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FOREWORD 

The  Training  Analysis  and  Evaluation  Group  (TAEG)  was  tasked  by  the 
Chief  of  Naval  Education  and  Training  (CNET)  to  conduct  a  study  to  determine 
the  relative  effectiveness  and  efficiency  of  individualized  instruction  for 
different  kinds  of  training  tasks  and  ability  levels  of  trainees.  A  series 
of  reports  is  being  prepared  to  document  different,  specific  aspects  of  the 
TAEG  program.  This  is  the  first  report  in  that  series.  It  deals  with  over¬ 
all,  inter school  analyses  of  training  effectiveness  and  efficiency  measures. 
Subsequent  reports  in  the  series  will  present: 

•  results  of  intraschool  analyses  of  training  effectiveness  and 
efficiency  measures 

•  a  critical  review  of  literature  relevant  to  issues  involved  in 
assessing  instructional  method  effects  on  students 

•  results  of  analytical  efforts  concerning  officer  courses  and  an 
enlisted  course  that  uses  both  individualized  and  conventional 
instruction. 
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SUMMARY  OF  THE  STUDY 

The  Chief  of  Naval  Education  and  Training  (CNET)  tasked  the  Training 
Analysis  and  Evaluation  Group  (TAEG)  to  assess  the  effectiveness  and  effi¬ 
ciency  of  individualized  instruction  (II)  relative  to  conventional  instruc¬ 
tion  (Cl).  To  the  extent  that  differences  in  training  effectiveness  and/or 
efficiency  were  found  to  exist,  the  further  purpose  of  the  study  was  to 
determine  if  these  differences  are  related  to  student  ability  levels  and/or 
to  specific  types  of  training  tasks. 

Two  measures  each  of  training  effectiveness  and  training  efficiency 
were  examined.  Student  end-of -course  grades  were  used  as  an  internal 
criterion  of  training  effectiveness.  Fleet  supervisor  ratings  of  the 
adequacy  of  training  for  identified  school  graduates  were  used  as  an 
external  criterion  of  effectiveness.  Training  efficiency  measures  were 
reflected  by  student  time  to  complete  training  and  by  an  individual  training 
cost  measure  derived  from  total  instructional  costs  for  a  given  course. 
Student  ability  levels  were  represented  by  Armed  Forces  Qualification  Test 
(AFQT)  percentiles  obtained  by  converting  their  Armed  Services  Vocational 
Aptitude  Battery  (ASVAB)  test  scores.  The  instructional  content  of  the 
various  courses  examined  during  the  study  was  equated  to  a  common  base  by 
classifying  learning  objectives  of  each  course  into  one  of  five  generic  task 
categories:  fact,  procedure,  category,  rule,  principle. 

Nineteen  Navy  "A"  schools  were  examined.  Ten  of  the  courses  were  con¬ 
ducted  using  II  formats  (eight  were  self-paced  courses,  two  featured 
computer  managed  instruction).  The  other  nine  courses  were  conducted  under 
conventional,  group-paced  instruction. 

A  correlational  approach  was  used  to  conduct  the  study.  Record  data 
were  collected  on  over  5,000  graduates  of  the  19  enlisted  technical  schools. 
The  results  of  all  statistical  analyses  performed  on  the  data  are  presented 
succinctly  in  section  IV  of  the  report.  Major  findings  and  conclusions  of 
the  study  are  summarized  below. 

1.  Individualized  instruction  (self-paced  and  computer  managed)  and 
conventional  group-paced  instruction  were  found  to  be  equally  effective  ways 
of  preparing  sailors  for  operational  fleet  jobs.  This  conclusion  is  based 
on  comparisons  of  fleet  supervisor  ratings  on  1,229  graduates  of  Cl  with 
ratings  on  1,186  graduates  of  individualized  courses. 

2.  Within  the  II  category,  self-paced  and  computer  managed 
instruction  were  found  to  be  equally  effective  training  methods  as  reflected 
by  fleet  supervisor  ratings. 

3.  Individualized  instruction,  as  it  was  conducted  during  the  time 
period  examined,  benefited  higher  ability  students  during  training  more  than 
It  did  lower  ability  students.  Relative  to  the  lower  ability  students, 
higher  ability  personnel  mastered  more  course  content  (i.e.,  had  higher  end- 
of  -course  grades)  and  completed  training  in  less  time. 

4.  No  definitive  evidence  was  obtained  concerning  which  method  of 
instruction  most  benefits  lower  ability  personnel. 
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5.  Conventional  instruction  did  not  benefit  one  ability  level  of  students 
over  another  during  training.  End-of -course  grades  for  conventionally  trained 
students  were  equivalent  across  all  ability  levels. 

6.  Fleet  supervisor  ratings  were  not  dependent  on  school  performance. 

No  significant  relationships  were  found  between  supervisor  ratings  and  graduate 
end-of -course  grades  or  time-to-complete  training. 

7.  Fleet  supervisor  ratings  did  not  depend  on  ability  level  or  sex  of 
the  graduate. 

8.  When  course  content  is  classified  into  generic  training  tasks,  II 
was  found  to  be  more  effective  than  Cl  in  courses  that  taught  primarily  proce¬ 
dure  tasks.  Conventional  instruction  was  more  effective  than  II  in  courses 
that  taught  primarily  rule  or  principle  tasks. 

9.  No  one  method  of  instruction  was  found  to  be  universally  more  effec¬ 
tive  in  training  all  of  the  different  types  of  tasks  to  different  ability 
level  students.  The  evidence  of  this  study  suggests  that  a  combination  of 
methods  used  within  a  given  course  for  conveying  different  instructional 
contents  would  likely  be  more  effective  than  use  of  a  single  method  for  an 
entire  course. 

10.  Definitive  conclusions  concerning  the  efficiency  of  II  relative  to 
Cl  could  not  be  reached  from  the  data  of  this  study. 

The  available  data  showed  that  overall  training  costs  were  higher  for 
the  II  courses  studied  than  for  the  Cl  courses.  However,  it  cannot  be  con¬ 
cluded  that  II  is  more  costly  than  Cl  since  different  courses  with  different 
contents  and  lengths  were  taught  under  each  method.  Similarly,  time  to  com¬ 
plete  training  was  found  to  be  less  overall  for  graduates  of  II  courses  than 
for  those  of  Cl  courses.  Again,  however,  because  of  differences  in  the  courses 
studied,  it  could  not  be  concluded  from  this  study  that  individualizing  instruc¬ 
tion  necessarily  reduces  time  required  for  training. 

Evidence  was  obtained,  however,  that  bears  on  the  time  reduction  issue. 
Mental  Category  1  and  2  personnel  required  significantly  less  time  to  complete 
training  under  II  than  Mental  Category  3  and  4  personnel  (under  II).  From 
this,  it  can  be  inferred  that  converting  current  Cl  courses  to  an  II  format 
would  likely  reduce  the  time  required  for  training  for  at  least  the  higher 
mental  category  groups  (although  the  average  course  completion  time  cannot 
be  specified).  This  inference  is  consonant  with  the  findings  of  previous 
research  which  assert  generally  that  student  time  is  reduced  when  courses 
are  converted  from  a  Cl  to  an  II  format--although  the  previous  research  did 
not  relate  the  reductions  to  particular  ability  groups. 

General  recommendations  are  made  within  the  report  concerning  the  use 
of  II  in  Navy  technical  training.  In  addition,  areas  are  identified  where 
additional  information  is  required  to  develop  a  more  complete  understanding 
of  factors  involved  in  assessing  and  improving  the  efficiency  and  effective¬ 
ness  of  II.  In  this  regard,  two  additional  work  efforts  have  been  initiated 
by  the  TAEG.  The  first  is  concerned  with  identifying  definitive  baseline 
estimates  of  average  student  time  to  complete  training  under  II.  The  second 
effort  is  concerned  with  improving  the  conduct  of  individualized  instruction 
within  the  NAVEDTRACOM. 
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SECTION  I 
INTRODUCTION 


BACKGROUND 


A  recent  review  of  the  status  of  individualized  instruction  (II)  within 
the  Naval  Education  and  Training  Command  (Zajkowski,  Heidt,  Corey,  Mew,  and 
Micheli,  1979)  identified  a  number  of  areas  in  which  additional,  detailed 
information  was  required  to  optimize  the  use  of  II  in  Naval  technical  train¬ 
ing.  One  recommendation  concerned  the  need  for  an  effort  "to  determine  the 
relative  effectiveness  and  efficiency  of  individualized  instruction  for  dif¬ 
ferent  kinds  of  training  tasks  and  ability  levels  of  trainees."  Knowledge 
of  the  existence  of  and  specific  nature  of  any  such  differential  relationships 
would  materially  assist  decision  makers  in  selecting  instructional,  strategies 
for  different  training  situations.  Subsequently  the  TAEG  was  tasked  by  CNETl 
to  conduct  the  study  recommended. 

PURPOSE  OF  THIS  STUDY 


The  purpose  of  the  present  study  was  to  assess  the  effectiveness  and 
efficiency  of  II  relative  to  conventional  instruction  (Cl).  To  the  extent 
that  differences  in  training  effectiveness  and/or  efficiency  were  found  to 
exist,  a  further  purpose  was  to  determine  if  these  differences  were  differ¬ 
entially  related  to  student  ability  levels  and/or  to  different  types  of 
training  tasks. 

STUDY  REPORTS 


A  number  of  reports  dealing  with  different,  specific  aspects  of  the 
TAEG  program  are  planned  for  publication.  These  are  identified  below. 

The  present  report  focuses  on  interschool  comparisons  to  address  the 
overall  objectives  of  the  study.  This  report  provides  information,  obtained 
from  studying  19  different  Navy  "A"  schools,  about  training  effectiveness 
and  efficiency  relative  to  the  major  variables  of  the  study;  i.e.,  method  of 
instruction,  student  ability  level,  and  type  training  task. 

Another  technical  report  will  provide  detailed  information  about  each 
school/course  involved  in  the  study  program.  The  major  focus  of  this  second 
report  is  on  intraschool  analyses  of  the  relationships  between  student  ability 
levels  and  measures  of  training  effectiveness  and  efficiency. 

A  selective  review  of  research  and  technical  literature  bearing  on  issues 
related  to  differential  training  effectiveness  will  be  presented  in  a  third 
report.  The  report  will  summarize  and  discuss  the  major  findings  of  the 
literature  and  describe  previous  approaches  used  to  assess  differential 
effectiveness. 


!CNET  ltr  Code  N-53  of  22  Apr  1980. 
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Two  of  the  courses  studied  during  the  work  program  were  taught  using 
self-paced  (SP)  instruction  at  one  location  and  group-paced  (GP)  instruction 
at  another.  One  of  these  courses,  Hull  Maintenance  Technician  (HT  "A"),  is 
also  dual-phased.  The  other.  Damage  Control  Assistant  (DCA),  is  an  officer 
course.  These  special  courses  required  statistical  treatments  different 
from  those  used  in  other  analyses.  Consequently,  the  results  concerning 
instructional  method  effects  on  training  effectiveness/efficiency  obtained 
with  these  courses  will  be  reported  separately  from  the  other  investigations. 

Another  planned  report  will  deal  with  the  procedures  used  to  equate  the 
content  of  the  different  technical  courses  to  a  common  base;  i.e.,  into  generic 
task  classifications.  It  will  also  present  information  concerning  differences 
in  classifications  made  by  school  subject  matter  experts  (SME)  and  research 
personnel.  The  results  of  comparisons  between  different  training  task  classi¬ 
fication  systems  will  also  be  presented. 

Finally,  an  executive  summary  of  the  entire  study  program  will  be  prepared. 
It  will  summarize  the  major  findings  of  the  study  and  provide  overall  study 
conclusions  and  recommendations. 

ORGANIZATION  OF  THIS  REPORT 

The  remainder  of  this  report  is  presented  in  five  sections  and  seven 
appendices.  Section  II  presents  the  technical  approach  of  the  study.  Section 
III  contains  the  results  concerning  the  evidence  for  differential  effective¬ 
ness  and  efficiency  as  related  to  the  major  variables  examined:  method  of 
instruction,  type  training  task,  and  student  ability  level.  A  discussion  of 
the  results  and  the  interpretations  that  can  be  made  are  given  in  section 
IV.  Conclusions  and  recommendations  are  presented  in  section  V.  Section  VI 
describes  additional  work  that  has  been  initiated  by  the  TAEG  in  the  II  area. 

Appendix  A  provides  a  concise  summary  of  previous  research  findings 
concerning  II  versus  Cl  used  in  military  settings.  Information  concerning 
the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  is  contained  in  appendix 
B.  ASVAB  scores  were  used  as  measures  of  student  ability  for  the  present 
study.  Appendix  C  presents  the  training  package  given  to  SMEs  for  classifying 
instructional  content  into  generic  task  categories.  Elements  of  the  cost 
data  base  used  in  assessing  training  efficiency  are  discussed  in  appendix  D. 
Appendix  E  identifies  the  schools  studied  during  the  work  program.  Appendix 
F  presents  a  description  of  the  hierarchical  regression  model  used  for  data 
analysis.  Information  obtained  from  analysis  of  missing  data  in  the  study 
is  provided  in  appendix  G. 
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SECTION  II 
TECHNICAL  APPROACH 

This  section  presents  the  technical  approach  to  the  study.  A 
perspective  on  the  study  is  presented  first.  This  is  followed  by  a  brief 
description  of  previous  research  concerning  the  effectiveness  and  efficiency 
of  II  versus  Cl.  Subsequently,  discussions  of  the  variables  examined,  the 
samples  of  courses  and  graduates  involved  in  the  present  study,  and  the 
analytical  design  of  the  study  are  provided. 

STUDY  PERSPECTIVE 

The  study  was  designed  to  answer  two  major  questions: 

1.  Are  significant,  relative  differences  in  training  effectiveness 
and/or  efficiency  associated  with  individualized  instruction  versus  conven¬ 
tional  instruction? 

2.  If  there  are  differences,  are  they  related  to  different  kinds  of 
training  tasks  and/or  ability  levels  of  trainees? 

Since  experimental  methods  were  impractical  to  employ  within  the 
context  of  an  ongoing  military  training  system,  a  correlational  approach  was 
used  to  conduct  the  study.  Under  this  approach,  statistical  analyses  are 
performed  on  record  data  to  determine  if  significant  relationships  exist 
among  variables  of  interest. 

Within  the  approach  selected,  the  procedural  steps  consisted  of: 

•  acquiring  background  information  and  knowledge 

•  selecting  relevant  variables  including  identifying  sources  of  data 
on  the  variables 

•  selecting  courses  taught  under  each  of  the  basic  instructional 
methods 

•  selecting  samples  of  graduates  of  the  courses 

•  collecting  appropriate  data 

•  analyzing  the  data. 

Each  of  these  topics  is  discussed  further  below. 

BACKGROUND 

At  the  beginning  of  the  program,  a  brief  literature  review  was 
conducted  to  obtain  background  information  for  the  study.  The  review 
focused  on  a  number  of  topics  and  issues  relevant  to  the  present 
investigation  including: 

•  findings  of,  and  methodologies  employed  in,  previous  studies 
conducted  in  both  academic  and  military  environments  concerning 
the  effects  on  training  effectiveness/efficiency  of  different 
instructional  methods 

•  various  definitions  and  meanings  of  the  concept  of  ability  and 
student  achievements  related  to  differing  ability  levels 
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•  types  of,  and  previous  uses  of,  learning  task  classification 
systems 

•  student  learning  of  different  types  of  instructional  content. 

Information  obtained  from  the  literature  review  was  used  in  designing 
aspects  of  the  present  study,  in  developing  study  lines  of  inquiry,  and  in 
selecting  the  analytical  strategy  used.  In  addition  to  these  uses,  the  body 
of  knowledge  available  in  the  literature  was  found  to  be  of  considerable 
general  interest.  Consequently,  a  separate  technical  report  is  being 
prepared  to  integrate  and  consolidate  the  information  contained  in  the 
various  sources. 

Of  immediate  and  direct  interest  to  the  present  TAE6  program,  however, 
were  those  studies  which  compared  the  use  of  II  versus  Cl  in  military 
settings.  Orlansky  and  String  (1979)  reviewed  these  efforts.  Their  review 
indicated  that: 

•  II  is  as  effective  as  Cl  in  terms  of  student  end-of -course 
achievement  scores 

•  the  efficiency  of  II  is  sinoif icantly  better  than  that  of  Cl  in 
terms  of  student  time  to  cCmplet:  instruction 

•  training  costs  for  II  are  i**ir  due  largely  to  student  time 
savings 

•  the  addition  of  computer  support  to  self-paced  instruction  does 
not  significantly  ir crease  student  time  savings. 

Additional  findings  of  the  Orlansky  and  String  review  are  presented  in 
appendix  A  of  this  report. 

Orlansky  and  String  noted  that  the  previous  studies  contrasting  II  and 
Cl  were  more  or  less  controlled  experimental  studies,  were  of  short 
duration,  and  used  small  numbers  of  subjects.  The  authors  cited  the  need 
for  information  about  the  effects  of  II  used  over  a  longer  period  of  time 
within  an  operational  training  context  where  situational  factors  differ 
considerably  from  those  involved  in  the  previous  research.  The  CNET  tasking 
of  TAEG  required  an  assessment  of  II  effects  within  Naval  Education  and 
Training  Command  (NAVEDTRACOM)  schools  where  both  methods  are  used  routinely 
for  diverse  skill  and  knowledge  training.  Thus,  the  present  TAEG  study 
complements  previous  research  by  assessing  the  generalizability  of  prior 
findings. 

STUDY  VARIABLES 

Seven  major  variables,  three  predictor  and  four  criterion  variables, ? 
were  selected  for  study.  The  variables,  the  measures  of  the  variables 


^Precise  definitions  of  these  terms  may  be  found  in  various  statistical 
sources;  for  example,  Finn  (1974)  and  Cohen  &  Cohen  (1975). 
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employed,  and  the  sources  of  data  on  the  variables  are  identified  in  table 
1.  Further  discussions  of  these  variables  and  data  sources  are  provided 
below. 

PREDICTOR  VARIABLES.  The  major  predictor  variables  were  method  of  instruction, 
trainee  ability  level,  and  type  training  task. 

Method  of  Instruction.  The  primary  predictor  variable  of  the  study  was  instruc¬ 
tional  method.  Two  basic  methods  were  of  interest:  individualized  instruction 
and  conventional  instruction.  Individualized  instruction  involved  self- 
paced  (frequently  referred  to  as  IMI  for  instructor -managed  instruction)  and 
computer -managed  courses  (CMI).  Conventional  instruction  involved  group- 
paced  (GP)  classroom  instruction. 

Ability  Level.  Ability  levels  of  trainees  constituted  the  second  major 
predictor  variable.  For  the  present  study,  ability  levels  of  trainees  were 
represented  by  student  scores  on  the  ASVAB.  This  test  is  routinely  adminis¬ 
tered  to  all  armed  services  enlistees. 

The  ASVAB  consists  of  12  subtests.  The  composition  of  the  battery  prior 
to  October  1980  is  shown  in  appendix  B.  Various  combinations,  or  composites, 
of  ASVAB  subtest  scores  are  used  by  the  Navy  to  determine  an  individual's 
eligibility  for  attendance  at  specific  technical  schools.  Scores  from  three 
of  the  battery  subtests  can  also  be  used  to  derive  an  Armed  Forces  Qualifica¬ 
tion  Test  (AFQT)  percentile  score  (see  appendix  B).  These  percentile  scores 
provide  a  measure  of  general  ability.  They  can  also  be  used  to  group  indivi¬ 
duals  into  mental  categories.  AFQT  percentile  scores  (derived  from  the  pre- 
October  1980  conversion  routines)  were  used  to  represent  ability  levels  of 
graduates  for  making  comparisons  across  courses.  A  subsequent  TAEG  report 
will  present  relationships  between  composite  scores  and  the  other  variables 
of  interest. 

Types  of  Training  Tasks.  The  third  predictor  variable  was  the  types  of  tasks 
trained  in  school.  Definitive  conclusions  concerning  the  value  of  II  versus 
Cl  for  training  different  types  of  tasks  require  that  the  same  tasks  be  taught 
under  each  method.  Unfortunately,  the  same  courses  (and,  consequently,  the 
same  tasks)  are  not  taught  under  each  of  the  two  basic  methods  of  instruction, 
and  it  was  beyond  the  scope  of  the  study  to  create  this  condition  experimental ly. 
Thus,  the  matching  of  tasks  under  different  methods  of  instruction  was  approxi¬ 
mated  by  using  a  generic  task  classification  system.  The  content  of  the 
different  courses  (i.e.,  tasks  taught)  was  equated  to  a  common  base  so  that 
appropriate  comparisons  could  be  made.  The  use  of  these  comparisons  assumes 
that  the  psychological  processes  involved  in  the  acquisition  of  generic  tasks 
are  similar  regardless  of  the  specific  context  in  which  the  learning  occurs. 

For  the  present  TAEG  study,  a  modification  of  the  Instructional  Quality 
Inventory  (IQI)  method  (Ellis,  Wulfeck,  &  Fredericks,  1979)  was  used  to  equate 
skill  and  knowledge  items  taught  in  each  of  the  courses  studied  to  a  common 
base.  Subject  matter  experts  at  each  school  classified  statements  of  the 
school's  learning  objectives  into  one  of  five  types  of  information  content: 
fact,  category,  procedure,  rule,  and  principle.  The  classifications  are 
described  in  table  2.  (See  appendix  C  for  a  description  of  the  training 
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TABLE  1.  MAJOR  PREDICTOR  AND  CRITERION  VARIABLES 


Variable  Measures  Source 


PREDICTOR 
Ability  Level 
Method  of  Instruction 
Type  of  Training  Task 

CRITERION 
EFFECTIVENESS 
End-of -Course  Grades 

Training  Adequacy 
Ratings 

EFFICIENCY 
Cost  of  Course 


Time  to  Complete 


AFQT,  Composites 
SP,  CMI,  GP 
Five  Generic  Tasks 


Final  Grades 

1-5  Scale 

Average  Cost  to 
Produce  One  Graduate 
(Per  Session) 

Contact  Hours 


CNET  015 
CNET,  CNTECHTRA 
IQI  (SMEs) 


School  Records (SP  +  GP), 
CNTECHTRA  (CMI) 

TAS  (CNET) 


CNET  Accounting  System 


CNTECHTRA  (CMI), 

School  Records, 

SMEs,  NITRAS  (SP  +  GP) 
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TABLE  2.  INSTRUCTIONAL  QUALITY  INVENTORY  CONTENT  TYPES 


Content  Type  is: 

Fact: 

If  the  student  must  recall  or  recognize  names,  parts, 
locations,  functions,  dates,  places,  etc.  Example:  name 
the  parts  of  the  — . 

Category: 

If  the  student  must  remember  characteristics  of  similar 
objects,  events,  or  ideas,  OR  if  the  student  must  sort  or 
classify  objects,  events,  or  ideas  according  to  character¬ 
istics.  Example:  identify  target  types  from  sonar 
signal  displays. 

Procedure: 

If  the  student  must  remember  a  sequence  of  steps  which 
apply  to  a  single  situation,  OR  if  the  student  must  apply 
the  steps  to  a  single  piece  of  equipment  or  a  single 
situation.  Example:  Field  strip  an  M-16  rifle. 

Rule: 

If  the  student  must  remember  a  sequence  of  steps  and 
decisions  which  apply  in  a  variety  of  situations,  OR  if 
the  student  must  apply  the  sequence  across  a  variety  of 
situations  or  types  of  equipment.  Example:  Use 
mathematical  formulas  such  as  Ohm's  law. 

Principle: 

If  the  student  must  remember  how  or  why  things  work  the 
way  they  do,  or  cause-effect  relationships,  OR  if  the 
student  must  use  his  knowledge  to  explain  how  things  work 
or  predict  effects  from  causes.  Example:  Based  on  knowl¬ 
edge  of  electronic  theory,  predict  effect  on  the 
circuit  if - . 

Source:  Ellis,  Wulfeck,  &  Fredericks  (1979) 
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package  given  to  the  SMEs  for  classifying  course  learning  objectives  using 
the  IQI  system.) 

In  addition  to  those  made  by  school  SMEs,  classifications  of  the  same 
school-trained  skill  and  knowledge  items  were  also  made  (at  TAEG  request)  by 
Navy  Personnel  Research  and  Development  Center  (NPRDC)  staff.  The  NPRDC 
staff  who  made  the  classifications  routinely  use  the  iQI  system  and  were 
involved  in  its  development.  Also,  the  questionnaire  items  were  classified 
using  a  second  task  classification  system,  TECEP  (see  Braby,  Henry,  Parrish, 

&  Swope,  1978),  to  assess  differences  between  classification  schemes.  Com¬ 
parative  analyses  of  differences  between  raters  using  the  same  classification 
system  and  differences  between  classif ication  systems  are  currently  being 
performed.  The  results  of  these  analyses  will  be  provided  in  a  later  TAEG 
report. 

Miscellaneous  Predictor  Variables.  In  addition  to  the  three  primary  predictor 
variables  discussed  above,  the  effects  of  two  other  variables,  sex  of  the 
graduate  and  geographic  location  of  the  school(s),  were  also  assessed. 

CRITERION  VARIABLES.  Two  measures  each  of  training  effectiveness  and  training 
efficiency  were  used  as  criterion  variables.  The  effectiveness  measures 
were  end-of -course  grades  and  fleet  supervisor  ratings  of  school  training 
adequacy.  The  efficiency  measures  were  time-to-complete  training  and  training 
costs. 

Training  Effectiveness  Measures.  End-of-course  grades  which  reflected  how 
well  graduates  performed  in  school  were  used  as  an  internal  measure  of 
training  effectiveness.  Fleet  supervisor  ratings  which  reflected  graduate 
job  performance  were  used  as  an  external  measure  of  course  effectiveness. 
Internal  measures  of  student  learning  achievement  were  available  at  the 
schools.  External  measures  of  student  learning  achievement  were  available 
at  CNET. 

The  external  measures  consisted  of  fleet  supervisor  ratings  of  the 
adequacy  of  school  training  for  particular  tasks  which  graduates  are 
expected  to  perform  on  the  job.  External  criteria  of  training  effectiveness 
were  of  special  interest  to  the  study.  The  Orlansky  and  String  (1979) 
review  of  II  versus  Cl  contrasts  in  military  settings  noted  that  previous 
studies  were  confined  to  comparisons  on  measures  of  student  learning 
achievement  available  at  the  schools  (see  appendix  A).  External  criteria  of 
training  effectiveness  were  apparently  not  used  (or  were  unavailable)  for 
assessing  the  effects  of  different  instructional  methods. 

Fleet  supervisors'  ratings  of  training  adequacy  on  identified  courses  and 
graduates  were  available  at  CNET  in  the  NAVEDTRACOM  Training  Appraisal 
System  (TAS)  data  base.  The  CNET  Special  Assistant  for  Training  Appraisal 
(CNET  015)  routinely  collects  feedback  data,  via  mailout  questionnaires, 
from  first-level  fleet  supervisors  of  recent  technical  school  graduates. 

Random  samples  of  graduates  are  drawn  from  the  total  pool  of  course 
graduates  during  a  given  time  frame. 

Table  3  shows  the  type  of  data  collected  on  school  graduates  that  was  of 
interest  to  the  study.  Fleet  supervisors  rate  on  a  5-point  scale  the 
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adequacy  of  school  training  for  an  identified  course  graduate.  Training 
adequacy  judgments  are  made  for  a  number  of  specific  tasks  for  which  a  given 
technical  school  provided  training.  The  task  statements  listed  on  a 
feedback  questionnaire  are  currently  prepared  by  technical  training  staff 
for  a  given  course.  The  statements  are  based  on  the  learning  objectives  of 
that  course  and  thus  reflect  specific  skills  and  knowledges  taught  at  a 
school . 


TABLE  3.  EXAMPLES  OF  TRAINING  APPRAISAL  SYSTEM  DATA 


Adequacy  of  School  Training: 

Circle  One  Number  for  Each  Item  Listed 

1.  Unsatisfactory 

2.  Less  Than  Adequate 

3.  Adequate 

4.  More  Than  Adequate 

5.  Much  More  Than  Adequate 


1  2  3  4  5 


1  2  3  4  5 

1  2  3  4  5 


1  2  3  4  5 

1  2  3  4  5 


Skill  or  Knowledge  Item 

(Fact)  Identify  Purposes  and  Organization  of 

Personnel  Qualification  Standards 

(Category)  Recognize  Symptoms  of  Severe  Electric 
Shock 

(Procedure)  Operate  Ship's  Store  Cash  Register 

(Rule)  Develop  Drawing  Layout  for  Sheet 

Metal  Projects 

(Principle)  Troubleshoot  Magnetic  Amplifiers 


The  task  statements  shown  in  table  3  are  typical  of  those  used  on  TAS 
questionnaires.  However,  the  particular  statements  iri  the  table  represent 
tasks  of  five  different  ratings.  They  were  deliberately  chosen  to  provide 
examples  of  the  five  generic  task  types.  The  classification  names  shown  in 
parentheses  in  the  table  do  not  appear  on  TAS  questionnaires. 

Listings  of  tasks  taught  at  each  school  studied  and  supervisor  ratings 
of  training  adequacy  for  these  tasks  were  obtained  from  CNET.  The  question¬ 
naire  task  statements  for  each  school  were  classified  by  appropriate  school 
SMEs  into  the  generic  task  categories  described  earlier.  This  procedure 
permitted  supervisor  training  adequacy  ratings  to  be  compiled  for  specific 
generic  tasks.  These  data  were  then  used  for  making  "task-type"  comparisons 
within  and  between  courses. 
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Two  different  criterion  measures  were  formed  from  the  TAS  fleet  super¬ 
visor  ratings  for  use  in  study  analyses.  One  measure  consisted  of  a  mean 
TAS  rating  for  each  graduate  involved  in  the  study.  Mean  TAS  ratings  (for 
each  graduate)  were  derived  from  the  sum  of  all  ratings  assigned  a  given 
graduate  divided  by  the  number  of  questionnaire  items  for  which  the  graduate's 
supervisor  assigned  a  rating.  The  second  measure  used  was  a  task-specific 
TAS  rating.  Task -specif ic  TAS  ratings  consisted  of  mean  ratings  for  each 
generic  task.  The  task-specific  ratings  were  derived  by  summing  the  ratings 
pertaining  to  a  given  generic  task  and  dividing  by  the  number  of  ratings 
pertaining  to  that  given  type  of  task. 

Training  Efficiency  Measures.  Measures  used  to  reflect  training  efficiency 
were  training  costs  and  student  time  (contact  hours  of  instruction)  to  com¬ 
plete  a  course. 

Training  cost  data  were  available  in  the  CNET  Per  Capita  Cost  Data  Base 
for  each  course  in  the  study.  The  training  cost  data  were  referenced  to 
fiscal  year  1979  which  coincided  with  the  time  period  that  most  graduates  of 
interest  to  the  study  were  in  training.  From  the  cost  data  base,  a  "training 
cost  to  produce  one  graduate  per  course  session"  was  derived  and  used  in 
subsequent  analyses.  Appendix  0  explains  the  derivation  of  training  cost 
data. 


Student  course  completion  times  were  available  for  individualized  courses 
from  CM I  files  and  from  class  records  at  the  schools  for  self-paced  courses. 
Completion  times  for  the  conventional  courses  were  obtained  from  the  schools 
and  from  CNET  computer  files. 

STUDY  SAMPLES 

Two  sets  of  samples  were  required  for  the  study.  The  first  consisted 
of  courses  taught  under  each  of  the  basic  methods  of  instruction,  the  second 
of  graduates  of  these  courses. 

COURSE  SAMPLES.  The  TAEG  project  staff  coordinated  with  CNET  and  Chief  of 
Naval  Technical  Training  (CNTECHTRA)  staff  to  select  courses.  The  plan  was 
to  identify  approximately  10  A-level  courses  that  used  Cl  ( i . e. ,  group- 
paced)  and  10  others  that  used  II  (i.e.,  self-paced  or  computer  managed). 
Courses  in  each  instructional  category  were  to  include  the  full  range  of 
ability  levels  of  individuals  who  undergo  Navy  technical  training.  It  was 
also  desired  that  courses  in  each  category  be  roughly  matched  on  general 
instructional  content  (i.e.,  type  training  tasks)  and  on  geographic  location. 

Initial  selection  of  courses  for  inclusion  in  the  method  of  instruction 
groups  was  made  on  the  basis  of  entries  in  the  Navy  Integrated  Training 
Resources  and  Administrative  System  (NITRAS).  This  system  identifies  courses 
that  are  considered  to  be  individualized  (SP  or  CMI)  and  those  considered  to 
be  taught  conventionally  (GP). 

Courses  selected  for  the  study  were  primarily  those  classified  as  A-level. 
A-level  courses  provide  basic  skill  and  knowledge  training  for  entry  level 
Navy  jobs.  These  courses  were  selected  rather  than  C-level  courses  because: 
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a  greater  diversity  of  tasks  is  trained  in  the  more  general 
A-level  courses  than  in  the  largely  equipment-specific  C-level 
courses 

a  wider  range  of  student  abilities  is  involved  in  A-level  traininq 
proportionately  more  A-level  courses  are  taught  under  II. 

It  was  further  desired  that  courses  be  selected  to  the  extent  possible 
from  those  for  which  TAS  data  were  already  available  or  soon  to  be  available 
(i.e.,  within  approximately  the  next  6  months).  Although  training  appraisal 
schedules  could  be  altered,  this  would  have  resulted  in  lengthy  time  delays 
to  obtain  data  on  course  graduates.  Consequently,  availability  of  the  TAS 
data  became  the  major  determinant  of  the  sample  composition.  It  is 
believed,  however,  that  the  other  criteria  were  reasonably  well  met  and  that 
the  courses  constituted  a  representative  sample  of  Navy  "A"  schools.  The 
course  samples  are  shown  in  table  4.  The  complete  names  of  the  courses  and 
school  locations  are  provided  in  appendix  E. 

Twenty-three  A-level  courses  were  included  in  the  overall  study.  Nine 
self-paced  and  two  CMI  courses  were  in  the  II  group.  Eleven  group-paced 
courses  were  in  the  Cl  group.  One  Hull  Technician  course  was  conducted 
using  mixed  instructional  methods.  Two  officer  courses  were  also  selected 
for  study.  The  content  of  the  courses.  Damage  Control  Assistant,  was 
ostensibly  the  same  but  each  course  was  taught  under  a  different  method  of 
instruction.  In  addition,  data  were  obtained  from  two  basic  CMI  courses. 
Graduates  of  the  RM  Sea  "A"  School  and  the  RM  Shore  "A"  School  also  attended 
the  RM  Basics  course.  Their  records  were  obtained  from  CNTECHTRA  CMI  files 
and  used  in  analyses  of  interest  to  the  study.  Similarly,  graduates  of  the 
EN,  MM  600  psi,  and  MM  1200  psi  schools  attended  the  Propulsion  Engineering 
(PE)  Basics  course  prior  to  entry  into  their  respective  "A"  schools.  Their 
records  were  also  obtained  from  CMI  files. 

GRADUATE  SAMPLES.  As  mentioned  previously,  the  names  of  school  graduates 
who  were  included  in  the  NAVEDTRACOM  TAS  graduate  samples  were  obtained  from 
CNET.  Table  5  shows  the  numbers  of  gradua'  s  for  each  school  for  whom  TAS 
records  were  obtained.  Over  all  of  the  schools  studied,  a  total  of  7,083 
graduate  records  were  examined  (5,278  for  the  schools  shown  in  the  table 
plus  1,805  records  for  those  school  graduates  who  first  attended  CMI  basic 
courses  before  entering  an  "A"  School).  The  table  also  shows  the  method  of 
instruction  used  at  each  school  and  the  inclusive  graduation  dates  for  the 
students.  The  numbers  of  graduates  of  each  school  for  whom  data  were  avail¬ 
able  on  variables  of  interest  are  also  shown.  Note  that  some  individualized 
courses  give  students  an  end-of -course  grade  while  others  do  not.  In  somp 
instances,  the  end-of-course  grade  is  given  based  on  a  comprehensive 
examination;  in  others,  it  represents  some  combination  of  module  test 
scores. 
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TABLE  4.  COURSE  SAMPLES 


Self-Paced 

Damage  Control  Ass't  (Phila)l 
Hull  Technician  II  (Phi la)  (HT) 
Machinist's  Mate,  600  psi3  (W16) 
Machinist's  Mate,  1200  ps i 3  (MM12) 
Instrumentman  (IM) 

Training  Device  Repairman  (TD) 
Personnelman  (PN) 

Yeoman  (YN) 

Disbursing  Clerk  ( DK ) 

Aviation  Storekeeper  (AK) 


CM  I 

Engineman?  (EN) 

Av'n  Machinist's  Mate  (AD) 


Group -Paced 

Damage  Control  Ass't  (T.I.)l  (DCA) 
Hull  Technician  I  (T.I.)  (HT) 

Hull  Technician  I  (Phila)  (HT) 
Radioman,  Sea?  (RM) 

Radioman,  Shore2  (RM) 

Electrician's  Mate  (EM) 

Fire  Control  Technician  (FT) 
Gunner's  Mate  (GM) 

Av'n  Support  Eq.  Tech  (ASE) 

Av'n  ASW  Operator  (AW) 

Ship's  Serviceman  (SH) 

Aviation  Electrician's  Mate  (AE) 

Mixed 

Hull  Technician  II  (S.D.)  (HT) 


lOff  icer  Course 

?Also  obtained  data  from  CMI  RM  basics  course 
^Also  obtained  data  from  CMI  PE  basics  course 
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TABLE  5.  GRADUATE  SAMPLES 


Course 

Type 

Graduating 

Dates 

Original 

Sample 

Size 

ASVAB 

T  ime 
to 

Complete 

End  of 
Course 
Grade 

Ratings 
on  TAS 
Items 

AK 

SP 

10/79-4/80 

254 

241 

231 

0 

170 

DCA  (Phila) 

SP 

10/79-3/80 

60 

0 

59 

54 

43 

DK 

SP 

8/79-2/80 

110 

107 

110 

110 

79 

HT2  (Phila) 

SP 

10/78-3/79 

319 

318 

294 

303 

218 

IM 

SP 

10/79-4/80 

18 

18 

17 

16 

15 

MM  600 

SP 

10/78-3/79 

374 

372 

284 

282 

138 

MM  1200 

SP 

9/78-3/79 

204 

203 

176 

176 

124 

PN 

SP 

11/78-5/79 

84 

74 

69 

0 

65 

TO 

SP 

1/79-7/79 

185 

179 

185 

0 

162 

YN 

SP 

11/78-5/79 

222 

208 

211 

0 

174 

AD 

CM  I 

10/79-4/80 

464 

450 

454 

464 

98 

EN 

CM  I 

3/79-6/79 

359 

356 

349 

244 

192 

HT2 

(San  Diego) 

MIXED 

10/78-4/79 

264 

260 

261 

190 

171 

AE 

GP 

3/79-5/79 

90 

86 

90 

90 

54 

ASE 

GP 

8/79-2/80 

36 

29 

36 

36 

29 
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TABLE  5.  GRADUATE  SAMPLES  (continued) 


Course 

Type 

Original 
Graduating  Sample 
Dates  Size 

ASVAB 

Time  End  of 

to  Course 

Complete  Grade 

Ratings 
on  TAS 

Items 

AW 

GP 

1/79-7/79 

47 

42 

45 

45 

35 

DCA 

GP 

10/79-4/80 

46 

0 

46 

46 

37 

(Treasure  Island) 

EM 

GP 

8/78-2/79 

362 

354 

349 

306 

262 

FT 

GP 

10/78-4/79 

77 

77 

66 

66 

34 

GM 

GP 

7/79-12/79 

47 

47 

47 

47 

21 

HT1 

GP 

10/78-3/79 

319 

318 

319 

0 

218 

(Phila) 

HT1 

GP 

10/78-4/79 

264 

260 

198 

197 

171 

(Treasure  Island) 

RM-Sea 

GP 

2/79-12/79 

496 

495 

487 

0 

369 

RM-Shore  GP 

3/79-8/79 

400 

397 

400 

0 

396 

SH 

GP 

10/79-3/80 

177 

176 

177 

177 

132 

Subtotals :SP 

10/78-4/80 

1830 

1720 

1636 

941 

1188 

CM  I 

3/79-4/80 

823 

806 

803 

708 

290 

Mixed  10/78-4/79 

264 

260 

261 

190 

171 

GP 

8/78-4/80 

2361 

2281 

2260 

1010 

1758 

Total 

8/78-4/80 

5278 

5067 

4960 

2849 

3407 
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DATA  COLLECTION 

The  names,  graduation  dates,  and  social  security  numbers  of  school 
graduates  were  obtained  from  NAVEDTRACOM  TAS  files.  Visits  were  made  to  the 
25  schools  between  August  and  November  1980  to  obtain  data  on  course  gradu¬ 
ates.  At  the  schools,  data  were  manually  recorded  from  class  records  and 
entered  on  worksheet  forms  for  subsequent  entry  into  computer  files.  Data 
recording  was  accomplished  either  by  TAEG  project  staff  or  school  SMEs  func¬ 
tioning  under  general  TAEG  supervision.  Information  recorded  consisted 
principally  of  end-of -course  grades  and  time-to-complete  training.  Where 
available,  other  measures  of  training  effectiveness/efficiency  were  also 
recorded.  These  other  measures  included  numbers  of  academic  remediations 
and  setbacks,  and  numbers  of  additional  hours  of  instruction  required. 

Training  cost  data  relevant  to  each  course  were  obtained  from  CNET. 

Training  adequacy  ratings  and  questionnaire  task  statements  were 
obtained  from  the  CNET  TAS  data  base.  The  data  included  the  fleet  super¬ 
visors'  TAS  ratings  for  each  graduate  (i.e.,  1,  2,  3,  4,  or  5)  on  each 
skill/knowledge  item  of  the  course  feedback  questionnaire.  The  12  ASVAB 
subtest  scores  of  each  graduate  were  also  obtained  from  the  CNET  TAS  data 
base  (or  the  student  master  file  when  necessary).  During  school  visits, 
assigned  SMEs  (3  to  5  at  each  school)  classified  the  items  on  the  TAS  ques¬ 
tionnaires  into  the  generic  task  categories.  This  was  done  under  general 
TAEG  supervision  and  under  the  guidance  presented  in  appendix  C.  Subject 
matter  expert  classifications  were  also  entered  into  the  data  base.  All 
data  collection  was  completed  by  mid-1981. 

DATA  ANALYSIS 

Analyses  of  training  effectiveness  and  training  efficiency  were  con¬ 
ducted  on  four  different  groupings  of  the  courses  in  the  study.  These 
groupings  are  shown  in  table  6. 

The  first  analysis  was  based  on  data  of  19  single-phase  enlisted  "A" 
schools.  The  purpose  of  this  analysis  was  to  investigate  possible  differ¬ 
ences  between  Cl  and  II  across  courses  as  well  as  to  assess  interrelation¬ 
ships  with  the  two  other  predictor  variables  (i.e.,  student  ability  level 
and  type  training  task). 

Data  concerning  two  dual-phase  enlisted  "A"  Schools  were  used  in  the 
second  analysis.  In  these  schools,  enlisted  personnel  received  a  different 
method  of  instruction  in  each  phase  of  the  "A"  School.  A  major  purpose  of 
the  second  analysis  was  to  investigate  the  possibility  of  transfer  effects 
between  methods  of  instruction  within  the  same  group  of  graduates. 

The  third  analysis  was  on  data  of  two  single-phase  officer  courses. 

Both  officer  courses  nominally  present  the  same  subject  matter  but  each  under 
a  different  method  of  instruction.  Thus,  the  purpose  of  the  third  analysis 
was  to  investigate  possible  differences  in  methods  of  instruction  between 
the  two  officer  courses  when  training  content  is  held  "constant." 

Data  obtained  from  all  the  courses  included  in  the  previous  three  analyses 
plus  the  two  basic,  pre-"A"  School  courses  were  used  for  the  fourth  analysis. 
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TABLE  6.  COURSES  INCLUDED  IN  THE  FOUR  ANALYSES 


First  Analysis  Second  Analysis  Third  Analysis  Fourth  Analysis 


Name 

N 

Type 

Name 

CDP 

N 

Type 

Name 

CDP 

N 

Type 

Name 

N  Type 

AD 

464 

CM  I 

HT2 

6339 

319 

SP 

DCA 

3214 

46 

GP 

All  courses 

in 

EN 

359 

CM  I 

HT1 

6120 

319 

GP 

previous  analyses 

plus 

AK 

254 

SP 

HT1 

6119 

264 

GP 

DCA 

3218 

60 

SP 

RM  Basics 

894  CM I 

DK 

110 

SP 

HT2 

6106 

264 

SP+GP 

PE 

Basics  (EN) 

359  CM I 

IM 

18 

SP 

PE 

Basics  (MM) 

552  CM I 

MM6  374  SP 

MM12  204  SP 

PN  84  SP 

TD  185  SP 

YN  222  SP 

AE  90  GP 

ASE  36  GP 

AW  47  GP 

EN  362  GP 

FT  77  GP 

GM  47  GP 

RMSE  496  GP 

RMSH  400  GP 

SH  177  GP 
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The  purpose  of  the  fourth  analysis  was  to  provide  an  in-depth  investigation 
of  training  efficiency  and/or  effectiveness  within  each  course  studied. 

The  present  report  addresses  only  the  results  of  the  first  analysis. 

The  results  of  the  other  analyses  will  be  presented  in  subsequent  TAEG  reports 
which  are  in  preparation. 

STATISTICAL  MODEL.  A  part  ial  hierarchical  regression  model  was  employed  to 
examine  the  effects  of  each  set  of  predictors  on  the  criterion  variables 
(Cohen  &  Cohen,  1975;  Kim  &  Kohout,  1975).  This  model  allowed  a  unique  par¬ 
titioning  of  the  total  variance  of  each  criterion  to  be  accounted  for  by 
each  subset  of  predictors  entered  into  the  regression  equation.  The  use  of 
a  multiple  regression  technique  is  consonant  with  current  methodological 
approaches  in  investigating  aptitude-treatment  interactions  (Cronbach  &  Snow, 
1977).  For  the  present  TAEG  study,  predictor  variables  were  considered  sta¬ 
tistically  significant  and  relevant  if:  (1)  they  met  an  acceptable  level  of 
significance  (p<.05)  and  (2)  provided  additional  explanation  of  the  variance 
of  the  criterion  variable  (increment  the  R^  by  2  percent).  For  each  criter¬ 
ion  variable,  an  overall  regression  analysis  was  first  performed.  Then, 
individual  comparisons  were  made  on  the  variables.  See  appendix  F  for  speci¬ 
fication  of  the  hierarchical  regression  model  used  in  the  study. 

MISSING  DATA  ANALYSIS.  Before  proceeding  with  specific  investigations  of 
relationships  among  the  variables  of  interest,  a  missing  data  analysis  was 
conducted.  This  was  done  to  determine  if  the  sample  of  II  and  Cl  graduates 
for  whom  criterion  data  were  available  differed  significantly  from  the  sample 
for  whom  criterion  data  were  absent.  Although  the  results  of  the  missing 
data  analysis  indicated  overall  differences  based  on  method  of  instruction, 
ability  level,  and  training  task,  no  significant  interactions  were  found 
(see  appendix  G).  Thus,  graduates  for  whom  data  on  end-of -course  grades, 
time-to-complete,  and  TAS  ratings  were  present  did  not  differ  in  any  combina¬ 
tion  of  background  characteristics  (e.g.,  ability  level  by  method  of  instruc¬ 
tion)  from  those  graduates  who  had  no  data  present  on  the  criterion  variables. 
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SECTION  III 
RESULTS 

The  results  of  the  study  are  provided  in  this  section.  All  results  are 
based  on  data  analyses  conducted  across  19  different  "Al"  schools/courses. 

Ten  of  the  courses  were  individualized  (eight  SP,  two  CMI)  and  nine  were 
group-paced.  The  specific  courses  examined  were  identified  in  table  6  (under 
the  column,  "first  analysis").  The  original  sample  sizes  (N)  of  graduates 
trained  under  each  instructional  method  are  given  in  table  7. 


TABLE  7.  ORIGINAL  SAMPLE  SIZES  OF  GRADUATES 


Method  of  Instruction  N 

Individualized  (SP  +  CMI)  2274 

Self-Paced  (SP)  1451 

Computer  Managed  Instruction  (CMI)  823 

Conventional  (GP)  1732 

Group-Paced  (GP)  1732 

Total  4006 


In  this  section,  a  relatively  large  number  of  tables  and  figures  are 
used.  These  are  necessary  to  display  adequately  the  study  findings  and  to 
present  the  supporting  statistical  evidence  for  the  many  significant 
relationships  observed  among  the  variables  of  interest.  All  significant 
findings  are  shown,  however,  in  three  tables  presented  at  the  end  of  the 
section  (tables  35,  36,  and  37).  In  the  main  body  of  the  section,  the 
results  are  organized  according  to  the  format  described  immediately  below. 

All  results  are  presented  in  order  of  the  criterion  variables  examined; 
i.e.,  the  measures  of  training  effectiveness  and  training  efficiency.  For 
each  criterion  measure,  tables  are  presented  to  summarize  the  significant 
results  of  overall  regression  analyses.  Each  summary  table  for  a  criterion 
measure  identifies  the  predictors  assessed  and  lists  the  regression  coeffi¬ 
cients  (B)  and  F-ratios  (£)  obtained.  The  summary  tables  also  show  the 
degrees  of  freedom  (OF)  involved  in  specific  comparisons  and  the  magnitude 
of  a  particular  R?  increment.  The  numbers  listed  under  "step"  in  the  sum¬ 
mary  tables  refer  to  the  order  in  which  that  particular  set  of  predictor 
variables  was  entered  into  the  regression  equation  (see  appendix  F). 

As  mentioned  above,  the  statistical  summary  tables  identify  the 
significant  results  relative  to  a  given  criterion  variable.  After  the 
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summary  tables,  each  significant  predictor,  both  main  effects  and  interactions, 
is  further  delineated  separately.  Supporting  data  for  the  significant  results 
are  presented  in  tables.  For  each  criterion  variable,  nonsignificant  results 
are  grouped  together  in  one  paragraph.  Note  that  in  the  main  effects  analyses, 
data  concerning  a  particular  criterion  variable  are  combined  over  all  courses 
and  over  all  graduates.  The  purpose  of  the  main  effects  analyses  is  to  assess 
the  contribution  to  criterion  variance  of  one  particular  predictor  variable 
(e.g.,  method  of  instruction,  trainee  ability).  For  the  present  study,  the 
interpretation  of  significant  main  effects  should  be  tempered  by  a  significant 
interaction  of  that  main  effect  with  another  predictor  variable. 

TRAINING  EFFECTIVENESS  MEASURES 

The  criterion  measures  of  training  effectiveness  were  end  of  course 
grades  and  TAS  ratings.  Two  variables  were  formed  from  the  TAS  ratings: 
overall  mean  TAS  ratings  and  task-specific  TAS  ratings. 

CRITERION:  END-OF-COURSE  GRADES.  End-of -course  grades  represent  the  average 
percent  correct  obtained  by  a  graduate  on  items  tested  in  the  school.  End- 
of  -course  grades  obtained  from  the  schools  were  based  on  either  combinations 
of  scores  from  module/lesson  examinations  administered  during  the  course  of 
training,  or  on  comprehensive  examinations  administered  at  the  end  of  training. 
The  overall  results  of  regression  analyses  of  end-of -course  grades  are  presented 
in  table  8.  Significant  and  nonsignificant  results  are  further  detailed 
below. 

Method  of  Instruction.  Graduates  of  individualized  courses  ( SP  +  CMI)  received 
higher  end  of  course  grades  than  Cl  (GP)  graduates.  SP  graduates  received  higher 
end  of  course  grades  than  CMI  graduates  (see  table  9). 

Ability  Level.  Across  all  II  and  Cl  courses,  graduates  with  higher  AFQT  percen- 
tile  scores  received  higher  end  of  course  grades  than  those  with  lower  scores 
(see  table  10). 

Method  of  Instruction  by  Ability  Level.  Individualized  instruction  (SP  +  CMI) 
graduates  at  the  higher  AFQT  percentiles  received  higher  end  of  course  grades 
than  Cl  (GP)  graduates  at  the  higher  percentiles.  Lower  ability  II  and  Cl  gradu¬ 
ates  received  similar  end  of  course  grades.  Conventional  instruction  graduates 
at  higher  AFQT  percentiles  received  end  of  course  grades  similar  to  those  of  Cl 
graduates  at  the  lower  AFQT  percentiles.  For  both  SP  and  CMI  graduates,  AFQT  was 
positively  related  to  end  of  course  grade.  Both  SP  and  CMI  graduates  had  similar 
positive  relationships  between  end  of  course  grades  and  ability  level  (see  figure 
1  and  table  11). 

Training  Task.  Across  II  and  Cl  courses,  end  of  course  grades  were  higher  in 
courses  that  taught  a  larger  percentage  of  cateqory,  procedure,  and  rule  tasks 
than  in  courses  that  taught  a  lesser  percentage  of  these  tasks.  End  of  course 
grades  were  higher  in  II  and  Cl  courses  that  taught  a  smaller  percentage  of  fact 
and  principle  tasks  than  in  courses  that  taught  a  larger  percentage  of  these 
tasks  (see  table  12). 
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TA8LE  8.  SIGNIFICANT  RESULTS  FROM  REGRESSION  ANALYSES  OF  END-OF-COURSE  GRADES 


Step  Variables 

B 

F 

DF 

- 52 - 

Increment 

Overall  £_ 

DF 

1  II  vs  Cl 

(Tl) 

-4.13 

376.72 

1/2041 

.2034 

260.64 

2/2041 

SP  vs  CM I 

(T2) 

-2.55 

169.41 

1/2041 

2  AFQT 

(A) 

.13 

188.95 

1/2036 

.2041 

140.29 

5/2036 

Fact 

(ID 

-.60 

175.31 

1/2036 

Category 

(12) 

.28 

151.58 

1/2036 

Procedure  (13) 

.13 

141.29 

1 /2036 

Rule 

(14) 

.10 

92.31 

1/2036 

3 

Tl  x  11 

2.27 

49.73 

1/2031 

.0831 

66.27 

5/2031 

Tl  x  13 

-.30 

133.95 

1/2031 

A  x  Tl 

-.04 

13.07 

1/2031 
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TABLE  9.  END-OF-COURSE  GRADES  BY  METHOD  OF  INSTRUCTION 


Method  of  Instruction 

Mean 

SD* 

N 

SP 

90.84 

5.51 

584 

CM  I 

85.74 

8.24 

692 

II  (SP  +  CMI) 

88.08 

7.56 

1276 

Cl  (GP) 

82.10 

6.72 

768 

♦Standard  Deviation 


TABLE  10.  END-OF-COURSE  GRADES  BY  AFQT  PERCENTILE  RANGE 
AND  MENTAL  CATEGORY 


AFQT 

Percentile  Range 

Mental  Category 

Mean 

SD 

N 

93-99 

1 

92.68 

6.51 

135 

65-92 

2 

87.45 

7.63 

673 

49-64 

3U 

84.55 

7.32 

737 

31-48 

3L 

83.58 

7.63 

436 

21-30 

4U 

81.87 

7.83 

21 

10-20 

4L 

79.50 

0.0 

1 
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TABLE  11. 

END-0F-C0URSE 

GRADES  BY  METHOD  OF 

INSTRUCTION 

AND  ABILITY 

LEVEL 

AFQT 

Percentile 

Range 

Mental 

Category 

Method 

of 

Instruction 

Mean 

SD 

N 

93-99 

1 

SP 

95.50 

3.04 

102 

CM  I 

92.69 

9.16 

2 

SP  +  CM I 

95.44 

3.17 

104 

GP 

83.41 

6.30 

31 

65-92 

2 

SP 

91.96 

4.95 

273 

CM  I 

90.38 

6.13 

140 

SP  +  CM I 

91.42 

5.43 

413 

GP 

81.33 

6.24 

260 

49-64 

3U 

SP 

87.29 

4.64 

146 

CM  I 

86.42 

7.99 

263 

SP  +  CM I 

86.73 

6.98 

409 

GP 

81.83 

6.80 

328 

31-48 

3L 

SP 

86.38 

4.79 

58 

CM  I 

82.81 

8.27 

253 

SP  +  CM I 

83.47 

7.86 

311 

GP 

83.84 

7.05 

125 

21-30 

41) 

SP 

0 

0 

0 

CM  I 

82.12 

7.95 

20 

SP  +  CM  I 

( 

82.12 

7.95 

20 

GP 

77.07 

0 

1 

10-20 

4L 

GP 

79.05 

0 

1 

34 


Technical  Report  117 


TABLE  12.  ENO-OF-COURSE  GRADES  8V  PERCENTAGE  OF  TASK  TRAINED 


Task 

Percent 

Mean 

SD 

N 

Fact 

1-15 

81.35 

6.66 

852 

16-30 

77.69 

4.79 

66 

Category 

1-15 

84.92 

5.32 

859 

16-30 

89.02 

5.52 

680 

Procedure 

1-50 

82.94 

5.03 

721 

51-100 

87.41 

6.30 

1323 

Rule 

1-50 

82.47 

6.21 

1237 

51-100 

86.08 

4.89 

110 

Principle 

1-50 

87.98 

4.82 

936 

51-100 

79.36 

5.35 

306 

Method  of  Instruction  by  Training  Task.  Individualized  instruction  graduates 
received  higher  grades  in  courses  that  taught  a  smaller  percentage  of  fact  tasks 
than  Cl  graduates  in  courses  that  taught  a  smaller  percentage  of  fact  tasks. 

There  were  no  data  points  from  II  courses  for  a  larger  percentage  of  fact  tasks 
on  which  to  base  a  comparison  with  Cl  courses.  Individualized  instruction 
graduates  received  higher  grades  in  courses  that  taught  a  larger  percentage  of 
procedure  tasks  than  II  graduates  who  were  taught  a  smaller  percentage  of 
procedure  tasks.  Conventional  instruction  graduates  received  similar  grades  in 
courses  that  taught  smaller  and  larger  percentages  of  procedure  tasks  (see  figure 
2  and  table  13). 


TABLE  13.  END-OF-COURSE  GRADES  BY  PERCENTAGE  OF  TYPE  OF 
TRAINING  TASK  AND  METHOD  OF  INSTRUCTION 


Task 

Percentage 

Range 

Method 

Mean 

SD 

N 

Fact 

1-10 

II 

83.08 

7.65 

453 

Cl 

79.39 

5.44 

399 

11-20 

II 

0 

0 

0 

Cl 

77.69 

4.80 

66 

Procedure 

1-50 

II 

85.65 

5.07 

126 

Cl 

82.37 

5.11 

595 

51-100 

II 

88.34 

7.04 

1150 

Cl 

81.19 

6.17 

173 
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MEAN 

END-OF- 

COURSE 

GRADES 


Note: 


Mean  data  points  based  on  less  than  five  cases  are  not  plotted. 


Figure  2.  Mean  End-of-Course  Grades  by  Method  of 
Instruction  and  Training  Task 
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\  Nonsignificant  Results.  The  interactions  of  ability  level  by  training  task, 

|  and  ability  level  by  training  task  by  method  of  instruction  did  not  signifi- 

*  cantly  predict  end-of -course  grades.  The  main  effects  for  sex  of  graduate 

|  and  geographic  location  of  school  were  also  not  significant. 

f 

CRITERION:  OVERALL  TRAINING  APPRAISAL  SYSTEM  (TAS)  RATINGS.  For  this 
analysis,  a  mean  (overall)  TAS  rating  was  computed  for  each  course  graduate. 
The  means  were  derived  from  supervisor  ratings  of  the  graduate  on  individual 
TAS  questionnaire  items.  Significant  results  obtained  from  the  overall 
regression  analyses  are  presented  in  table  14.  Specific  findings  are  listed 
below. 


TABLE  14.  SIGNIFICANT  RESULTS  FROM  REGRESSION  ANALYSES  OF  OVERALL 
TRAINING  ADEQUACY  RATINGS 


Step 

Variables 

B 

F 

DF 

R2 

Increment 

Overall  £ 

DF 

1 

II  vs  Cl  (Tl) 

-0.16 

62.36 

1/2412 

.0267 

33.10 

2/2412 

2 

Fact  (11) 

0.01 

4.23 

1/2407 

.0378 

19.46 

5/2407 

Category  (12) 

-0.02 

81.74 

1/2407 

Method  of  Instruction.  Individualized  instruction  (SP  +  CMI)  graduates  received 
higher  mean  TAS  ratings  than  Cl  (GP)  graduates.  There  were  no  significant 
differences  between  SP  and  CMI  graduates  on  mean  TAS  ratings  (see  table  15). 


TABLE  15.  OVERALL 

TAS  RATINGS 

BY  METHOD  OF 

INSTRUCTION 

Method  of  Instruction 

Mean 

SD 

N 

SP 

2.90 

.70 

899 

CMI 

2.96 

.57 

287 

II  (SP+CMI) 

2.91 

.67 

1186 

Cl  (GP) 

2.69 

.67 

1229 
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Training  Task.  Across  II  and  Cl  courses,  graduates  who  attended  those  courses 
that  taught  a  smaller  percentage  of  fact  or  category  tasks  received  higher 
average  TAS  ratings  than  graduates  who  were  taught  a  greater  percentage  of 
these  tasks  (see  table  16). 


TABLE  16.  TAS  RATINGS  BY  TYPES  OF  TASKS 


Percentage 
of  Task 

Task 

Taught 

Mean 

SO 

N 

Fact 

1-10 

2.86 

.56 

408 

11-20 

2.73 

.64 

34 

Category 

1-10 

2.92 

.66 

1027 

11-20 

2.66 

.69 

1149 

21-30 

2.73 

.64 

34 

Nonsignificant  Results.  All  interactions  involving  combinations  of  ability 
level,  method  of  instruction,  and  training  task  did  not  significantly  predict 
mean  TAS  ratings.  The  following  main  effects  were  also  nonsignificant:  ability 
level,  end-of -course  grade,  time-to-complete  the  course,  sex  of  graduate, 
and  geographic  location  of  school. 

CRITERION:  TASK-SPECIFIC  TAS  RATINGS.  TAS  ratings  were  also  grouped  by 
items  classified  into  generic  tasks  (i.e.,  facts,  categories,  procedures, 
rules,  and  principles).  Then,  mean  TAS  ratings  were  computed  for  each  of 
the  generic  task  groupings.  These  groupings  and  resulting  means  permitted 
task-specific  TAS  ratings  to  be  regressed  on  a  specific  task,  and  on  ability 
level,  method  of  instruction,  and  their  interactions.  One  purpose  of  this 
analysis  was  to  determine  if  the  percentage  of  a  particular  task  trained  in 
"A"  School  was  related  to  the  ratings  obtained  on  TAS  items  classified  as 
that  particular  task.  This  permitted  answering  questions  of  the  form,  "Are 
higher  TAS  ratings  on  items  classified  as,  e.g.  fact  tasks,  associated  with 
a  larger  percentage  of,  e.g.  fact  tasks,  taught  in  'A'  School?"  This  analysis 
would  clarify  if  specific  training  content  in  the  school  setting  transfers 
to  the  fleet  (operational)  setting  (as  reflected  in  the  task-specific  TAS 
ratings).  Another  purpose  was  to  assess  the  interactions  of  specific  tasks 
with  trainee  ability  level  and  method  of  instruction. 

Significant  results  of  the  regression  analyses  for  all  task-specific  TAS 
ratings  are  presented  in  table  17.  Then,  the  results  of  these  analyses  are 
further  delineated  below  according  to  the  generic  task  to  which  the  TAS  ratings 
apply  (criterion  variables). 

Criterion:  Fact  TAS  Ratings.  Percentage  of  fact  task  trained,  AFQT 
percentile,  method  of  instruction,  and  the  interactions  of  these  variables 
did  not  significantly  predict  fact  TAS  ratings. 
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TABLE  17.  SIGNIFICANT  RESULTS  FROM  REGRESSION  ANALYSES  OF 
TASK-SPECIFIC  TAS  RATINGS 


Criterion 

Step 

Variable 

B 

F 

OF 

R2 

Increment 

Overall  F 

OF 

Category 

2 

Category  (12) 

-.02 

55.20 

1/1703 

.0309 

27.58 

2/1703 

Procedure 

3 

T1  x  13* 

-.01 

75.53 

1/2336 

.0334 

27.73 

3/2336 

Rule 

3 

T1  x  14** 

A  x  12*** 

-.02 

.01 

31.20 

5.01 

1/228 

1/228 

.0297 

12.64 

3/1228 

Principle 

1 

.15 

19.14 

1/871 

.0215 

19.14 

1/871 

*T1  x  13  =  Method  of  instruction  by  procedure  task  interaction. 
**T1  x  14  =  Method  of  instruction  by  rule  task  interaction. 

***A  x  T2  =  AFQT  by  method  of  instruction  (SP  vs  CMI)  interaction. 
****71  =  II  vs  Cl. 

Note:  No  significant  results  for  fact  TAS  ratings. 


Criterion:  Category  TAS  Ratings. 

Percentage  of  Category  Task  Trained.  Across  II  and  Cl  courses,  graduates 
who  were  presented  with  a  smaller  percentage  of  category  tasks  during  train 
ing  received  higher  category-specific  TAS  ratings  than  graduates  presented 
with  a  larger  percentage  of  category  tasks  during  training  (see  table  18). 


TABLE  18.  CATEGORY-SPECIFIC  TASK  RATINGS  BY 

PERCENTAGE  OF  CATEGORY  TASKS  TRAINED 


Percentage  of 

Category  Task 

Mean 

SD 

N 

1-15 

3.10 

.63 

1055 

16-30 

2.70 

.63 

583 

Nonsignificant  Results.  AFQT  percentiles,  method  of  instruction,  and  their 
interactions  with  percentage  of  category  task  trained  did  not  significantly 
predict  category  TAS  ratings. 
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Criterion:  Procedure  TAS  Ratings. 

Method  of  Instruction  by  Percentage  of  Procedure  Task  Trained.  Conventional 
instruction  graduates  who  were  presented  with  a  smaller  percentage  of  proce¬ 
dure  tasks  during  training  received  higher  procedure-specific  TAS  ratings 
than  II  graduates  presented  with  a  smaller  percentage  of  procedure  tasks. 

On  the  other  hand,  II  graduates  who  were  presented  with  a  larger  percentage 
of  procedure  tasks  during  training  received  higher  procedure-specific  TAS 
ratings  than  Cl  graduates  presented  with  a  larger  percentage  of  procedure 
tasks  during  training  (see  figure  3  and  table  19).  Percentage  of  procedure 
task  taught  was  positively  related  to  procedure  TAS  ratings  for  II  graduates 
and  negatively  related  for  Cl  graduates.  There  were  no  significant  differences 
between  SP  and  CMI  graduates  on  procedure  TAS  ratings. 


MEAN 

PROCEDURE- 

SPECIFIC 

TAS 

RATING 


Figure  3.  Mean  Procedure  TAS  Ratings  by  Percentage  of  Procedure 
Task  Taught  and  Method  of  Instruction 
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TABLE  19.  PROCEDURE-SPECIFIC  TAS  RATINGS  BY  PERCENTAGE  OF 

PROCEOURE  TASK  TRAINED  AND  METHOD  OF  INSTRUCTION 


Percentage  of 
Procedure  Task 

II 

Cl 

Mean 

SD 

N 

Mean 

SD 

N 

1-50 

2.71 

.70 

209 

3.00 

.73 

462 

51-100 

2.95 

.68 

920 

2.55 

.63 

753 

Nonsignificant  Results.  AFQT  percentile,  percentage  of  procedure  tasks 
trained,  method  of  instruction,  and  their  remaining  interactions  did  not 
significantly  predict  procedure-specif ic  TAS  ratings. 

Criterion:  Rule  TAS  Ratings. 

Method  of  Instruction  by  Percentage  of  Rule  Task  Trained.  Individualized 
instruction  graduates  who  were  presented  with  a  smaller  percentage  of  rule 
tasks  during  training  received  higher  rule-specific  TAS  ratings  than  Cl  gradu¬ 
ates  presented  with  a  smaller  percentage  of  rule  tasks  during  training.  On 
the  other  hand.  Cl  graduates  who  were  presented  with  a  larger  percentage  of 
rule  tasks  during  training  received  higher  rule-specific  TAS  ratings  than  II 
graduates  presented  with  a  larger  percentage  of  rule  tasks  during  training. 
Percentage  of  rule  task  was  negatively  related  to  rule  TAS  ratings  for  II 
graduates  and  positively  related  for  Cl  graduates  (see  figure  4  and  table 
20).  There  were  no  significant  differences  between  SP  and  CMI  graduates  on 
Rule  TAS  ratings. 

Method  of  Instruction  by  Ability  Level.  Higher  ability  CMI  graduates  received 
higher  rule-specific  TAS  ratings  than  higher  ability  SP  and  GP  graduates. 
Average  to  lower  ability  SP  and  GP  graduates  received  rule-specific  TAS  ratings 
similar  to  those  received  by  average  to  lower  ability  CMI  graduates.  There 
were  no  significant  differences  between  II  (SP  and  ''ombined)  and  Cl  (GP) 
graduates  on  rule-specific  TAS  ratings  by  ability  level  (see  figure  5  and 
table  21). 

Nonsignificant  Results.  Percentage  of  training  task,  AFQT  percentiles, 
method  of  instruction,  and  their  remaining  interactions  did  not  significantly 
predict  rule-specific  TAS  ratings. 

Criterion:  Principle  TAS  Ratings. 

Method  of  Instruction.  Conventional  instruction  (GP)  graduates  received 
higher  principle-  specific  TAS  ratings  than  II  (SP  and  CMI)  graduates. 

There  were  no  significant  differences  between  SP  and  CMI  graduates  on 
principle-specific  TAS  ratings  (see  table  22). 

Nonsignificant  Results.  Percentage  of  principle  tasks  trained,  ability 
level,  and  their  interactions  did  not  significantly  predict  principle- 
specific  TAS  ratings. 
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MENTAL  CATEGORY 


Note: 


21-30  31-48  49-64  65-92  93-99 

AFQT  PERCENTILES 


Mean  data  points  based  on  less  than  five  cases  are  not  plotted. 


Figure  5.  Mean  Rule  TAS  Rating  by  Ability  Level  and 
Method  of  Instruction. 
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TABLE  21.  RULE-SPECIFIC  TAS  RATINGS  BY  ABILITY  LEVEL 
AND  METHOD  OF  INSTRUCTION 


AFQT 

Percentile 

Mental 

Category 

Mean 

SP 

SD 

N 

Mean 

CM  I 

SD 

N 

Mean 

GP 

SD 

N 

93-99 

1 

2.84 

.70 

47 

3.45 

.93 

11 

2.89 

.72 

11 

65-92 

2 

2.87 

.74 

220 

3.10 

.78 

15 

2.75 

.73 

188 

49-64 

3U 

2.93 

.74 

218 

2.85 

.81 

24 

2.84 

.70 

235 

31-48 

3L 

2.92 

.86 

107 

3.0 

— 

1 

2.86 

.72 

92 

21-30 

4U 

2.93 

.45 

7 

-- 

-- 

— 

3.0 

-- 

1 

L0-20 

4L 

— 

— 

— 

— 

— 

— 

4.0 

— 

1 

TABLE  22.  PRINCIPLE-SPECIFIC  TAS  RATINGS 
BY  METHOD  OF  INSTRUCTION 


Method  of 

Instruction 

Mean 

SD 

N 

II  (SP  +  CMI) 

2.59 

CO 

• 

384 

Cl  (GP) 

2.82 

.67 

489 

TRAINING  EFFICIENCY  MEASURES 

The  measures  of  training  efficiency  examined  were  time  to  complete  the 
course  and  training  costs.  The  results  obtained  concerning  training 
efficiency  are  presented  below. 

CRITERION:  TIME  TO  COMPLETE  THE  COURSE.  Time  to  complete  the  course  was 
represented  by  student  contact  hours.  Student  contact  hours  reflect  the 
amount  of  time  the  student  was  under  direct  instruction.  Significant 
results  of  the  regression  analyses  performed  are  summarized  in  table  23. 

The  results  are  further  detailed  below. 
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TABLE  23.  SIGNIFICANT  RESULTS  FROM  REGRESSION  ANALYSES  OF 
TIME  TO  COMPLETE  (CONTACT  HOURS)  COURSES 


Variables 


AFQT  (A)  -  0.70  73.86 
Fact  (II)  12.89  700.34 
Category  (12)  -  5.96  1012.49 
Procedure  (13)  -  1.81  536.40 
Rule  (14)  -  0.79  67.98 


1/3789 

1/3789 


1/3784 

1/3784 

1/3784 

1/3784 

1/3784 


Increment  Overall  F 


86.30  2/3789  ! 


.4903  3979.38  5/3784  : 


T1  x  II 
T1  x  12 
T1  x  13 
T1  x  14 
A  x  T1 
A  x  T2 


27.47 

9.56 

3.42 

3.93 

1.00 

0.47 


318.38 

304.58 

268.62 

311.11 

85.24 

19.69 


1/3778 

1/3778 

1/3778 

1/3778 

1/3778 

1/3778 


163.67  6/3778 


Method  of  Instruction.  Individualized  instruction  (SP  +  CMI)  graduates  com¬ 
pleted  their  courses  in  less  time  than  Cl  (GP)  graduates.  Within  the  II 
group,  CMI  graduates  completed  their  courses  in  less  time  than  SP  graduates 
(see  table  24). 


TABLE  24.  TIME  TO  COMPLETE  THE  COURSE  (CONTACT  HOURS) 
BY  METHOD  OF  INSTRUCTION 


1 - - 

Method  of  Instruction 

Mean 

SD 

N 

SP 

166.35 

69.60 

1283 

CMI 

139.57 

88.25 

803 

II  (SP+CMI) 

156.04 

82.57 

2086 

Cl  (GP) 

195.22 

122.05 

1707 

Ability  Level.  Across  II  and  Cl  courses,  graduates  with  higher  AFQT  percen¬ 
tile  scores  finished  their  courses  in  less  time  than  those  with  lower  AFQT 
percentile  scores  (see  table  25).  Graduates  at  the  mid-AFQT  percentiles  had 
similar  course  completion  times.  These  main  effect  findings  are  tempered, 
however,  by  the  interaction  analysis  given  below. 
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TABLE  25 

.  TIME  TO  COMPLETE 
BY  ABILITY  LEVEL 

THE  COURSE 

(CONTACT  HOURS) 

AFQT 

Percentile  Range 

Mental  Category 

Mean 

SD 

N 

93-99 

1 

134.64 

93.05 

203 

65-92 

2 

178.19 

113.29 

1114 

49-64 

3U 

174.11 

103.45 

1551 

31-48 

3L 

172.12 

91.30 

800 

21-30 

41) 

191.73 

83.40 

34 

10-20 

4L 

480.00 

0.0 

1 

Method  of  Instruction  by  Ability  Level.  Individualized  instruction  graduates 
in  the  upper  mental  categories  finished  their  courses  in  less  time  than  cor¬ 
responding  Cl  graduates.  Both  II  (SP  +  CMI)  and  Cl  (GP)  graduates  in  the 
mid  and  lower  mental  categories  took  about  the  same  amount  of  time  to  complete 
their  courses  (see  figure  6  and  table  26).  Training  time  was  negatively 
related  to  ability  level  for  II  graduates,  and  positively  related  for  Cl 
graduates.  Training  time  was  significantly  less  for  CMI  graduates  than  for 
SP  graduates  at  each  ability  level  except  for  the  lowest  ability  level.  SP 
graduates  had  significantly  less  training  time  than  CMI  graduates  at  the 
lowest  ability  level  (mental  category  411). 

Training  Task.  Across  II  and  Cl  courses,  student  completion  times  were  longer 
for  courses  that  taught  a  greater  percentage  of  fact  or  principle  tasks 
than  a  lower  percentage  of  these  tasks.  Student  completion  times  were  shorter 
in  courses  that  taught  a  lower  percentage  of  category,  principle,  or  rule 
tasks  than  a  higher  percentage  of  these  tasks  (see  table  27). 

Method  of  Instruction  by  Training  Task.  Individualized  instruction  graduates 
had  lower  completion  times  in  courses  that  taught  a  smaller  percentage  of 
fact,  category,  procedure,  rule,  or  principle  tasks  than  Cl  graduates  taught 
a  smaller  percentage  of  these  tasks.  In  addition,  II  graduates  had  lower 
completion  times  in  courses  that  taught  a  larger  percentage  of  category  or 
procedure  tasks  than  Cl  graduates  taught  a  larger  percentage  of  these  tasks. 
For  II  graduates,  course  completion  time  was  negatively  related  to  the  per¬ 
centage  of  category  or  procedure  tasks  taught.  There  was  no  significant 
relationship  between  course  completion  time  and  percentage  of  rule  tasks 
taught  for  II  graduates.  For  Cl  graduates,  course  completion  time  was  posi¬ 
tively  related  to  the  percentage  of  fact  tasks  taught,  but  negatively  related 
to  the  percentage  of  category,  procedure,  or  principle  tasks  (see  figure  7  and 
table  28). 
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Note:  Mean  data  points  based  on  less  than  five  graduates  are  not  plotted 


Figure  6.  Mean  Time  to  Complete  the  Course  by  Method  of 
Instruction  and  Ability  Level 
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TABLE  26.  TIME  TO  COMPLETE  (CONTACT  HOURS)  BY  METHOD  OF 
INSTRUCTION  AND  ABILITY  LEVEL 


AFQT 

Percentile 

Mental 

Category 

Method 

of 

Instruction 

Mean 

SD 

N 

93-99 

1 

SP 

101.50 

48.72 

152 

CM  I 

83.33 

38.67 

6 

SP  +  CM  I 

100.83 

48.39 

158 

GP 

253.31 

113.53 

45 

65-92 

2 

SP 

149.52 

81.99 

508 

CM  I 

115.65 

61.08 

167 

SP  +  CM I 

141.14 

78.71 

675 

GP 

235.16 

133.08 

439 

49-64 

3U 

SP 

199.59 

82.33 

390 

CM  I 

130.13 

67.02 

310 

SP  +  CM I 

168.83 

33.37 

700 

GP 

178.45 

117.29 

851 

31-48 

3L 

SP 

198.59 

104.86 

178 

CM  I 

161.25 

69.87 

281 

SP  +  CM I 

175.73 

86.98 

459 

GP 

167.25 

96.73 

341 

21-30 

4U 

SP 

155.20 

43.06 

10 

CM  I 

188.53 

77.69 

21 

SP  +  CM I 

177.78 

69.51 

31 

GP 

335.87 

90.18 

3 

10-20 

4L 

GP 

480.0 

- 

1 
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CONTACT 

HOURS 


Figure  7.  Mean  Time  to  Complete  the  Course  by  Method  of 
Instruction  and  Type  Training  Task 
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TABLE  28.  TIME  TO  COMPLETE  (CONTACT  HOURS)  BY  PERCENTAGE  OF 
TASK  TAUGHT  AND  METHOD  OF  INSTRUCTION 


Percentage  of 
Task  Taught 

II 

Cl 

Task 

Mean 

SD 

N 

Mean 

SD 

N 

Fact 

1-15 

190.12 

50.46 

454 

282.16 

39.84 

442 

16-30 

0 

0 

0 

428.03 

50.53 

66 

Category 

1-15 

160.71 

43.15 

1197 

207.77 

55.56 

1062 

16-20 

73.84 

13.46 

348 

163.14 

16.70 

609 

Procedure 

1-50 

219.72 

45.39 

312 

281.97 

33.52 

638 

51-100 

131.86 

36.94 

1773 

143.45 

10.31 

1069 

Rule 

1-50 

194.72 

52.42 

982 

306.90 

33.13 

811 

51-100 

196.12 

35.28 

295 

0 

0 

0 

Principle 

1-50 

133.34 

43.00 

662 

342.73 

22.38 

462 

51-100 

0 

0 

0 

259.46 

43.45 

349 

Nonsignificant  Results.  The  interactions  of  ability  level  by  training  task, 
and  ability  level  by  training  task  by  method  of  instruction;  main  effects  of 
sex  of  graduate  and  geographic  location  of  school  did  not  significantly  pre¬ 
dict  time  to  complete  the  course. 

CRITERION:  TRAINING  COSTS.  To  conduct  analyses  of  instructional  costs,  an 
individual  training  cost  was  computed  for  each  graduate.  This  was  derived 
by  dividing  a  graduate's  course  completion  time  by  average  course  completion 
time  and  then  multiplying  this  quotient  by  the  "average  course  training  cost 
to  produce  one  graduate  per  course  session"  (see  appendix  C).  Computing 
individual  training  costs  in  this  manner  provided  the  variance  needed  within 
courses  to  conduct  regression  analyses  with  training  costs  as  the  criterion 
variable.  If  a  graduate's  course  completion  time  was  longer  than  the  average 
course  completion  time,  then  his/her  training  cost  would  be  greater  than  the 
average  course  training  cost.  If  a  graduate's  course  completion  time  was 
shorter  than  the  average  course  completion  time,  then  his/her  training  cost 
would  be  less  than  the  average  course  training  cost.  Thus,  the  computation 
of  individual  training  costs  associated  shorter  course  completion  times  for 
a  graduate  with  less  training  costs. 

The  overall  results  of  the  regression  analyses  of  training  costs  are 
presented  in  table  29.  Significant  and  nonsignificant  results,  both  main 
effects  and  interactions,  are  listed  below. 

Method  of  Instruction.  Significantly  higher  training  costs  were  associated 
with  II  (SR  +  CMI)  graduates  than  with  Cl  (GP)  graduates.  Training  costs 
were  significantly  greater  for  SP  graduates  than  for  CMI  graduates.  Train¬ 
ing  costs  were  similar  for  Cl  (GP)  and  CMI  graduates  (see  table  30).  The 
apparent  contradiction  of  higher  training  costs  and  lower  training  times 
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TABLE  29.  SIGNIFICANT  RESULTS  FROM  REGRESSION  ANALYSES  OF  TRAINING  COSTS 


Slt*p 

Predictor 

B 

F 

OF 

R2 

Increment 

Overall  F 

OF 

1 

II  vs  Cl 

(Tl) 

-115.97 

18.64 

1/3789. 

.0293 

57.17 

2/3789 

SP  vs  CM I 

(T2) 

-246.45 

80.70 

1/3789 

2 

AFQT 

(A) 

-  10.85 

98.38 

1/3784 

.3753 

471.98 

5/3784 

Fact 

(ID 

109.08 

275.09 

1/3784 

Category 

(12) 

-  58.05 

526.27 

1/3784 

Procedure 

(13) 

-  24.98 

561.56 

1/3784 

Rule 

(14) 

-  15.57 

146.52 

1/3784 

3 

AxTl 

13.94 

97.70 

1/3778 

.1177 

153.10 

1/3778 

A  x  T2 

4.75 

11.94 

1/3778 

T1  x  11 

233.45 

134.60 

1/3778 

T1  x  12 

132.12 

340.22 

1/3778 

T1  x  14 

23.22 

63.49 

1/3778 

4 

TICOM* 

11.75 

15418.92 

1/3769 

.3819 

15419.18 

1/3769 

♦Time  to  complete  the  course. 
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being  associated  overall  with  II  graduates  is  further  explained  by  the 
interaction  of  method  of  instruction  by  ability  level. 


TABLE  30. 

TRAINING  COSTS 

BY  METHOD  OF  INSTRUCTION 

Method  of  Instruction 

Mean 

SD 

N 

SP 

$  2573.71 

1102.36 

1283 

CMI 

2080.81 

705.78 

802 

II  (SP  +  CMI) 

2384.11 

969.21 

2085 

Cl  (GP) 

2153.31 

1467.29 

1707 

Ability  Level.  Across  II  and  Cl  courses,  training  costs  were  significantly 
greater  for  lower  mental  category  graduates  than  for  higher  mental  category 
graduates  (see  table  31).  Training  costs  were  similar  for  graduates  at  the 
middle  mental  categories.  This  main  effect  is  also  partitioned  further  by 
the  interaction  of  method  of  instruction  by  ability  level. 


TABLE  31.  TRAINING  COSTS  BY  ABILITY  LEVEL 


AFQT  Percentile 
Range 

Mental 

Category 

Mean 

SD 

N 

93-99 

1 

$1982.57 

1056.01 

203 

65-92 

2 

2370.01 

1285.24 

1114 

49-64 

3U 

2207.48 

1253.92 

1551 

31-48 

3L 

2285.47 

1136.47 

800 

21-30 

4U 

2837.38 

945.76 

34 

10-20 

4L 

4704.67 

0 

1 

Method  of  Instruction  by  Ability  Level.  Training  costs  were  significantly 
greater  for  mid  to  lower  mental  category  II  (SP  +  CMI)  graduates  than  for 
mid  to  lower  mental  category  Cl  (GP)  graduates.  On  the  other  hand,  training 
costs  were  significantly  greater  for  upper  mental  category  Cl  (GP)  graduates 
than  for  upper  mental  category  II  (SP  +  CMI)  graduates.  Also,  training 
costs  were  significantly  greater  for  mid  to  lower  mental  category  SP 


53 


Technical  Report  117 


graduates  than  for  mid  to  lower  mental  category  CMI  graduates.  Training 
costs  were  similar  for  upper  mental  category  SP  and  CMI  graduates.  Training 
costs  were  negatively  related  to  ability  level  for  II  graduates  and  positively 
related  for  Cl  graduates  (see  figure  8  and  table  32). 

The  explanation  for  the  apparent  contradiction— higher  training  costs 
but  shorter  training  time  for  II  graduates  when  longer  training  time  is 
related  to  higher  training  costs  over  all  courses— is  inherent  in  the  nature 
of  the  training  cost  variables  used.  When  training  costs  are  computed  for 
individual  graduates  across  all  courses,  students  who  take  more  time  to  com¬ 
plete  their  courses  do  indeed  cost  more  to  train  than  those  who  take  less 
time  to  complete  the  course.  This  is  due  to  the  relationship  of  the  average 
course  costs  being  multiplied  by  the  graduate's  individual  training  time 
divided  by  the  average  course  completion  time.  The  result  of  this  individual 
training  cost  metric  is  that  slower  students  are  assigned  a  training  cost 
value  greater  than  the  average  course  cost  and  faster  students  are  assigned 
a  value  lower  than  the  average  course  cost.  Thus,  the  individual  training 
costs  as  computed  in  this  study  allow  one  to  observe  a  positive  relationship 
overall  between  training  time  and  training  cost.  However,  the  average  course 
costs  represent  given  values  from  the  CNET  Per  Capita  Cost  Data  Base  and,  as 
such,  may  or  may  not  be  as  dependent  on  course  completion  time  as  the  indi¬ 
vidual  training  costs.  The  higher  costs  observed  for  the  II  courses  in  this 
study  may  be  due  to  a  course  characteristic  difference  that  may  or  may  not 
be  related  to  method  of  instruction.  One  way  to  clarify  this  issue  would  be 
to  obtain  and  analyze  the  cost  data  of  particular  categories  that  make  up 
the  total  training  costs  for  each  course  in  the  study.  The  results  of  this 
investigation  could  reveal  the  relative  contributions  of  instructional  method, 
specific  course  cost  categories,  and  their  interactive  effects  to  the  total 
training  costs. 

Type  Training  Task.  Across  II  and  Cl  courses,  training  costs  were  signifi¬ 
cantly  higher  for  courses  that  taught  a  greater  percentage  of  fact  or 
principle  tasks  than  in  courses  that  taught  a  lower  percentage  of  fact  or 
principle  tasks.  Training  costs  were  significantly  higher  in  courses  that 
taught  a  smaller  percentage  of  category,  procedure,  or  rule  tasks  than  in 
courses  that  taught  a  larger  percentage  of  category,  procedure,  or  rule  tasks 
(see  table  33). 

Method  of  Instruction  by  Training  Task.  Individualized  instruction  graduates 
had  lower  training  costs  in  courses  that  taught  a  smaller  percentage  of  fact, 
rule,  or  principle  tasks  than  Cl  graduates  in  courses  that  taught  a  smaller 
percentage  of  these  tasks.  Individualized  instruction  graduates  had  higher 
training  costs  in  courses  that  taught  a  smaller  percentage  of  category  tasks 
than  Cl  graduates  in  courses  that  taught  a  smaller  percentage  of  category 
tasks.  II  graduates  had  lower  training  costs  in  courses  that  taught  a  larger 
percentage  of  category  tasks  than  Cl  graduates  in  courses  that  taught  a  lower 
percentage  of  category  tasks.  Training  costs  were  not  significantly  related 
to  the  percentage  of  procedure  tasks  taught  by  method  of  instruction  (see 
figure  9  and  table  34). 
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Figure  8.  Mean  Training  Costs  By  Method  of  Instruction 
and  Ability  Level 


Technical  Report  117 


TABLE  33.  TRAINING  COSTS  BY  PERCENT  OF  TASKS  TRAINED 


Task 

Percent 

Mean 

SD 

N 

Fact 

1-15 

$2995.44 

575.07 

896 

16-30 

4196;78 

495.43 

66 

Category 

1-15 

2360.07 

665.94 

2259 

16-20 

1535.53 

214.71 

957 

Procedure 

1-50 

3086.84 

471.40 

950 

51-100 

2010.59 

872.19 

2842 

Rule 

1-50 

3093.06 

651.84 

1793 

51-100 

2755.93 

478.72 

295 

Principle 

1-50 

2761.42 

519.50 

1124 

51-100 

2958.58 

495.42 

349 

TABLE  34.  TRAINING  COSTS  BY  TRAINING  TASK  AND  METHOD  OF  INSTRUCTION 


Task 

Percent 

II  (SP  +  CMI) 

Cl  (GP) 

Mean 

So 

N 

Mean 

ib 

N 

Fact 

1-15 

$2488.37 

660.89 

454 

$3516.28 

471.64 

442 

16-30 

N/A 

N/A 

N/A 

4196.78 

495.43 

66 

Category 

1-15 

2484.52 

487.06 

1197 

2226.65 

615.82 

1062 

16-30 

1549.10 

282.22 

348 

1937.12 

164.11 

609 

Procedure 

1-50 

2958.62 

439.66 

312 

3149.54 

423.30 

638 

51-100 

2283.01 

717.72 

1773 

1558.74 

44.80 

1069 

Rule 

1-50 

2863.95 

810.68 

982 

3370.48 

378.65 

811 

51-100 

2755.93 

370.72 

295 

N/A 

N/A 

N/A 

Principle 

1-50 

2119.20 

656.14 

602 

3681.63 

502.08 

462 

51-100 

N/A 

N/A 

N/A 

2958.60 

495.42 

349 
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Figure  9.  Mean  Training  Costs  By  Method  of  Instructi 
and  Type  Training  Task 
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Time  to  Complete  the  Course.  In  general,  individual  training  costs  were 
greater  for  graduates  who  took  a  longer  time  to  complete  their  courses  than 
for  graduates  who  completed  their  courses  in  less  time  (R2  increment  =  .38). 
But  this  was  to  be  expected  because  training  costs  were  computed  using 
training  time  as  a  principal  basis. 

Nonsignificant  Results.  The  interactions  of  ability  level  by  training  task, 
and  ability  level  by  training  task  by  method  of  instruction;  main  effects  of 
sex  of  graduate  and  geographic  location  of  school  did  not  significantly 
predict  training  costs. 

SlfMARY  OF  FILINGS 

Summaries  of  the  findings  of  the  study  are  presented  next  in  three 
tables. 

The  first  table  (table  35)  lists  the  predictor  variables  assessed 
against  four  criterion  variables:  training  time,  training  costs,  end-of- 
course  grades,  and  overall  TAS  ratings.  Significant  and  nonsignificant 
results  are  indicated  in  the  cells.  The  "direction"  of  significant  results 
(e.g.,  whether  II  has  "better"  effects  than  Cl)  is  too  complex  for  simple 
summarization.  Consequently,  directionality  must  be  read  from  the  relevant 
portions  of  the  preceding  text. 

Table  36  summarizes  results  of  predictor  comparisons  by  five  generic 
task-specific  TAS  ratings  used  as  criterion  variables.  Significant  and 
nonsignificant  comparisons/results  are  indicated.  Again,  however,  the 
direction  of  effect  must  be  obtained  from  the  text. 

The  final  table  (table  37)  summarizes  the  salient  findings  of  the  study 
with  respect  to  the  effectiveness  and  efficiency  of  training  as  a  function 
of  II  versus  Cl. 
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TABLE  35.  SUMMARY  TABLE  OF  STATISTICAL  RESULTS  FROM  REGRESSION  ANALYSES  OF 
FOUR  CRITERION  VARIABLES  BY  PREDICTOR  AND  CRITERION  VARIABLES 


Criterion 

Variables 

Predictors 

Training 

Time 

Training 

Costs 

End-of- 

Course 

Grades 

Overall 

TAS 

Ratings 

( 

Method  of  Instruction  (MOI): 

II  vs  Cl 

X 

X 

X 

X 

i 

SP  vs  CM I 

X 

X 

X 

- 

Ability  (AFQT  (A)) 

X 

X 

X 

- 

MOI  by  A 

X 

X 

X 

- 

Percentage  of  Training  Task  (T): 

Fact 

X 

X 

X 

X 

Category 

X 

X 

X 

X 

Procedure 

X 

X 

X 

- 

Rule 

X 

X 

X 

- 

Principle 

X 

X 

X 

- 

T  by  A 

- 

- 

-• 

- 

T  by  MOI 

X 

X 

X 

- 

T  by  MOI  by  A 

- 

- 

- 

- 

Sex  of  Graduate 

- 

- 

- 

- 

Geographic  School  Location 

- 

- 

- 

- 

Training  Time 

N/A 

X 

— 

*  Nonsignificant  results 
X  =  Significant  results 

N/A  =  Not  applicable  to  analysis  of  specific  criterion 
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TABLE  36.  SUMMARY  TABLE  OF  STATISTICAL  RESULTS  FROM  REGRESSION  ANALYSES  OF 
TASK-SPECIFIC  TAS  RATINGS  BY  PREDICTOR  AND  CRITERION  VARIABLES 


Criterion 
(Task-Specific  TAS 

Ratinqs) 

Predictors 

Fact 

Category 

Procedure  1 

Rule  Principle 

Method  of  Instruction  (MOI): 

II  vs  Cl 

X 

SP  vs  CM I 

- 

- 

- 

- 

Ability  (AFQT  (A)} 

_ 

_ 

_ 

_  _ 

MOI  by  A 

- 

- 

- 

X 

Percentage  Training  Task  (T) 

- 

X 

- 

- 

T  by  A 

_ 

_ 

_ 

_  _  | 

T  by  MOI 

- 

- 

X 

x 

T  by  MOI  by  A 

- 

- 

-  -  1 

-  -  Nonsignificant  results 
X  =  Significant  results 


TABLE  37.  SUMMARY  OF  SALIENT  FINDINGS  OF  PERFORMANCE  MEASURES  OF  II 
GRADUATES  COMPARED  TO  Cl  GRADUATES 
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Higher  ability  II  graduates  completed  their  courses 
in  less  time  than  lower  ability  II  graduates. 
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SECTION  IV 
DISCUSSION 

The  results  of  the  study  are  discussed  in  this  section.  Brief  discus¬ 
sions  of  the  overall  efficiency  and  effectiveness  of  II  compared  to  Cl  are 
presented  first.  Discussions  of  the  interrelationships  of  II  with  trainee 
ability  levels  and  with  the  different  types  of  generic  training  tasks  are 
then  presented. 

OVERALL  EFFICIENCY 

Based  on  the  measures  used  in  this  study,  no  one  method  of  instruction 
was  found  clearly  to  be  more  efficient  than  another.  Training  efficiency— 
measured  in  terms  of  training  time  and  training  costs--was  independent  of 
fleet  measures  of  training  effectiveness.  Student  completion  time  and  cost 
to  produce  a  graduate  were  not  reliable  indicators  of  the  level  of  training 
adequacy  reported  in  the  fleet  setting  for  either  II  or  Cl  graduates. 

At  the  school  level,  the  finding  that  training  costs  were  higher  when 
producing  graduates  in  less  time  under  II  than  under  Cl  may  not  be  due  pri¬ 
marily  to  the  method  of  instruction.  Rather,  it  may  be  due  to  differences 
in  the  factors  comprising  the  cost  data  bases  for  specific  courses.  An 
analysis  of  the  subcomponents  of  training  costs  of  II  and  Cl  courses  would 
appear  to  be  an  appropriate  follow-up  study  into  the  reasons  why  training 
costs  are  different  overall.  Similarly,  training  time  differences  could  be 
approached  in  terms  of  what  aspects  of  courses  under  II  or  Cl  contribute 
significantly  to  student  time  savings.  Further  investigations  of  the 
efficient  aspects  of  conventional  instruction,  self-paced  instruction  with¬ 
out  computer  support,  and  self-paced  with  computer  support  may  even  lead  to 
an  acceptable  mixture  of  methods  for  use  within  a  given  course. 

OVERALL  EFFECTIVENESS 

Fleet  supervisor  ratings  of  school  training  adequacy  were  used  as  the 
external  criterion  of  training  effectiveness.  On  this  criterion,  the  II 
group  was  rated  significantly  higher  than  the  Cl  group.  Although  statis¬ 
tically  significant,  the  mean  values  of  ratings  for  both  groups  of  graduates 
fell  within  the  same  range  of  values— a  range  that  would  be  considered  to 
reflect  satisfactory  school  training.  Thus,  the  inference  can  be  made  that 
the  two  basic  methods  of  instruction  are  equally  effective  in  preparing 
graduates  for  operational  jobs.  Within  the  II  group,  graduates  of  CMI  courses 
were  rated  similarly  to  graduates  of  SP  courses.  Accordingly,  the  two  forms 
of  II,  SP  and  CMI,  can  be  considered  equally  effective  from  the  data  of  this 
study. 

For  the  internal  criterion  of  training  effectiveness--end-of -course 
grades--the  study  results  showed  significantly  higher  average  end-of -course 
grades  to  be  associated  overall  with  II.  However,  this  result  must  be  inter¬ 
preted  in  terms  of  the  interaction  between  end-of -course  grades  and  student 
ability  levels.  This  interaction  is  discussed  in  greater  detail  below. 
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ABILITY  LEVEL  OF  GRADUATES  AND  INDIVIDUALIZED  INSTRUCTION 

The  discussions  below  concern  how  graduates  of  different  ability  levels 
were  affected  by  II  and  cover  both  school  and  fleet  setting  criteria. 

SCHOOL  LEVEL.  At  the  school  level,  II  benefited  higher  ability  graduates 
more  than  lower  ability  graduates.  The  higher  ability  II  graduates  finished 
their  courses  sooner  and,  consequently,  were  less  costly  to  train  than  lower 
ability  II  graduates.  The  higher  ability  II  graduates  also  achieved 
significantly  higher  end-of -course  grades.  This  finding  is  consistent  with 
previous  research  which  has  shown  positive  relationships  between  student 
ability  and  school  achievement  (see  Cronbach  &  Snow,  1977). 

It  is  interesting  to  note  that  those  II  courses  providing  end-of -course 
grades  to  students  used  normative  (based  on  a  group)  measures  of  performance 
rather  than  an  all-or-none  (go/no-go)  mastery  criterion  which  is  typically 
used  in  an  II  format.  The  use  of  this  normative  criterion  in  determining 
student  readiness  to  graduate  apparently  did  not  influence  fleet 
supervisors'  perceptions  of  training  adequacy  for  these  graduates.  Fleet 
supervisor  ratings  were  not  related  to  end-of -course  grades. 

No  definitive  conclusions  concerning  training  time  and  training  costs 
could  be  drawn  for  the  different  ability  groups  trained  under  Cl.  For  this 
group,  the  higher  ability  graduates  tended  to  be  concentrated  in  courses  of 
greater  length.  Consequently,  over  all  Cl  courses,  higher  training  costs 
were  associated  with  higher  ability  Cl  graduates  than  with  lower  ability 
graduates.  This  reflects  an  apparent  selection  bias  which  may  have  operated 
to  place  higher  ability  students  in  longer  courses. 

No  significant  relationships  were  found  between  end  of  course  grades 
for  the  different  ability  levels  trained  under  Cl.  End  of  course  grades  for 
the  higher  ability  Cl  groups  were  essentially  the  same  as  those  for  the 
lower  ability  Cl  groups. 

The  failure,  in  this  study,  to  find  a  significant  relationship  between 
ability  levels  and  grades  for  Cl  graduates  is  interesting  since  previous 
research  has  documented  a  positive  relationship  between  these  two  variables 
(Cronbach  &  Snow,  1977).  Perhaps  something  is  occurring  within  the  Cl  con¬ 
text  that  obscures  or  restricts  the  potential  achievement  of  higher  ability 
personnel.  One  possibility  is  that  the  grading  system  used  is  not  suffici¬ 
ently  sensitive  to  discriminate  among  students  of  different  abilities. 
Another  is  that  the  Cl  context  contains  elements  that  adversely  affect  moti¬ 
vation  of  higher  ability  students  to  excel  much  beyond  the  minimum  school 
performance  criterion.  For  example,  the  higher  ability  Cl  graduates  may 
have  been  exposed  to  a  level  of  training  content  designed  for  the  majority 
(i.e.,  mid  to  lower  ability  level)  of  Cl  graduates  and  this  material  may 
have  posed  a  limited  challenge  to  the  more  capable  students.  Additionally, 
the  higher  ability  Cl  graduates'  motivation  may  be  affected  by  their  inabil¬ 
ity  to  control  the  amount  of  time  they  will  spend  in  the  Cl  course. 

To  recapitulate,  higher  ability  II  graduates  mastered  relatively  more 
course  content  than  lower  ability  II  graduates  and  did  so  in  significantly 
less  time.  For  the  Cl  graduates,  however,  both  end-of -course  grades  and 
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time  to  complete  training  were  similar  for  all  ability  groups.  These  find¬ 
ings,  taken  together,  suggest  that  a  Cl  course  converted  to  an  II  format  would 
provide  a  training  context  in  which  the  school  performance  of  different  ability 
level  students  is  differentiated.  For  the  newly  converted  course,  training 
costs  and  training  time  would  be  predicted  to  be  lower,  and  grades  higher, 
for  higher  ability  students.  The  average  course  completion  time  and  the 
amount  of  improvement,  however,  can  not  be  specified  from  the  data  of  the 
present  study.  An  issue  for  decision  makers  would  be  to  weigh  the  costs  and 
benefits  of  sending  a  highly  able,  trained  graduate  to  the  fleet  sooner, 
against  the  effort  involved  in  converting  Cl  courses  to  II  courses. 

There  were  no  definitive  findings  in  the  present  study  to  suggest  whether 
lower  ability  students  are  more  effectively  and/or  efficiently  trained  overall 
in  II  or  Cl  courses.  One  point  to  consider  here  concerns  the  frequently 
made  assertion  that  II  provides  a  context  in  which  lower  ability  students 
can  achieve  the  same  level  of  mastery  of  course  content  as  higher  ability 
students.  The  lower  ability  student,  however,  would  presumably  require  greater 
time  to  reach  this  level.  An  underlying  tenet  of  II  is  that  students  can 
proceed  through  training  at  their  own  pace.  However,  those  responsible  for 
training  may  have  determined  that  necessary  additional  time  for  course  mastery 
could  not  be  permitted  within  current  operational  constraints.  Hence,  a 
tradeoff  to  be  made  concerns  whether  the  probable  increase  in  grades  for 
lower  ability  students  is  worth  the  additional  cost  (i.e.,  greater  training 
time)  that  would  be  required  to  bring  all  students  to  an  equal  level  of  mastery. 

FLEET  SETTING.  School  performance  measures,  both  time  to  complete  training 
and  end-of -course  grades,  were  not  related  to  fleet  supervisor  ratings  of 
overall  training  adequacy  of  II  and  Cl  graduates  on  the  job.  Also,  general 
ability  measures  were  not  related  to  fleet  supervisor  ratings.  These  findings 
may  reflect  those  of  previous  research  which  has  reported  on  the  difficulty 
in  demonstrating  relationships  among  graduate  ability,  school  performance 
measures,  and  measures  reflecting  job  performance.  The  failure  to  find  rela¬ 
tionships  has  been  particularly  noted  when  supervisor  ratings,  rather  than 
job  knowledge  or  job  sample  tests,  are  used  as  indices  of  job  performance 
(e.g.,  Vineberg,  Sticht,  Taylor,  &  Caylor,  1971;  Wevrick,  1981).  Supervisor 
ratings  provide  a  greater  opportunity  for  the  occurrence  of  bias  than  do  the 
other  measures.  Thus,  rater  bias  is  one  possible  explanation  for  the  failure 
to  find  significant  relationships  among  the  variables.  Another  is  that  the 
instruments  and  procedures  used  for  obtaining  supervisor  ratings  are  not 
sufficiently  precise  or  sensitive  enough  to  detect  whatever  differences  may 
exist. 

Evidence  was  obtained  in  the  present  study,  however,  that  student  learn¬ 
ing  of  different  types  of  instructional  content  may  be  affected  by  the  method 
of  instruction  used.  Differences  were  observed  on  the  fleet  measures  when 
school  training  content  and  items  rated  by  fleet  supervisors  were  classified 
by  generic  tasks  trained  in  school.  These  findings  are  discussed  below. 

GENERIC  TRAINING  TASKS  AND  INDIVIDUALIZED  INSTRUCTION 

The  discussion  below  concerns  observed  relationships  between  generic 
training  tasks  and  graduate  performance  at  the  school  level  and  in  the  fleet 
setting. 
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SCHOOL  LEVEL.  School  performance  of  both  II  and  Cl  graduates  was  differen¬ 
tially  related  to  the  amount  of  different  types  of  generic  tasks  taught  in 
school.  Across  the  II  and  Cl  courses,  graduates  of  those  courses  that  taught 
larger  amounts  of  category,  procedure,  or  rule  tasks  cost  less  to  train, 
finished  their  courses  sooner,  and  received  higher  grades  than  graduates  of 
courses  featuring  smaller  amounts  of  these  tasks.  These  same  outcomes  (i.e., 
lower  cost,  less  time,  and  higher  grades)  were  also  observed  for  those  II 
and  Cl  courses  that  taught  smaller  amounts  of  fact  or  principle  tasks  compared 
to  courses  that  taught  larger  amounts  of  "hese  tasks.  Fleet  jobs  for  which 
students  were  required  to  learn  relatively  ;:-ore  fact  or  principle  tasks  in 
school  than  category,  procedure,  or  rule  tasks  required  longer  training  times. 
In  addition,  students  achieved  less  (i.e.,  got  lower  course  grades)  in  those 
courses.  Conversely,  courses  in  which  training  featured  greater  amounts  of 
category,  procedure  or  rule  tasks  tended  to  be  shorter  and  student  achieve¬ 
ment  was  higher. 

The  overall  (i.e.,  across  all  courses  examined)  relationships  found  for 
generic  training  tasks  were  not  differentiated  by  method  of  instruction  or 
by  trainee  ability  level.  Directional  changes  in  school  performance  measures 
as  a  joint  function  of  generic  task  type  and  instructional  method  were  incon¬ 
sistent.  In  addition,  no  evidence  was  obtained  in  the  present  study  to  suggest 
that  school  performance  of  different  ability  level  graduates  was  affected  in 
any  systematic  manner  by  different  amounts  of  generic  training  tasks  taught 
in  the  schools  sampled. 

FLEET  SETTING.  For  the  fleet  setting,  significant  differences  were  observed 
in  overall  training  adequacy  ratings  for  fact  or  category  tasks.  Graduates 
taught  smaller  amounts  of  fact  or  category  tasks  in  II  and  Cl  courses  received 
higher  fleet  supervisor  ratings  for  overall  training  adequacy  than  graduates 
taught  larger  amounts  of  these  tasks.  These  findings  are  similar  to  those 
found  for  fact  tasks  at  the  school  level.  However,  they  are  opposite  in 
diretion  to  the  relationship  observed  for  category  tasks  at  the  school  level. 
These  findings  suggest  that  smaller  amounts  of  fact  tasks  are  easier  to  train 
and  apply  on  the  job  than  larger  amounts  of  fact  tasks.  The  difference  between 
school  and  fleet  performance  findings  for  category  tasks  indicates  a  lack  of 
consistency  about  which  no  further  interpretation  was  made. 

When  the  TAS  items  rated  by  fleet  supervisors  were  classified  by  generic 
tasks  trained  in  both  II  and  Cl  courses,  II  was  found  to  be  more  effective 
than  Cl  in  courses  that  taught  primarily  procedure  tasks.  Conventional 
instruction  was  more  effective  than  II  in  courses  that  taught  primarily  rule 
or  principle  tasks.  These  results  suggest  that  training  tasks  concerned 
primarily  with  generalized  or  cause-effect  relationships  (rules  or  principles) 
may  be  more  adequately  presented  or  learned  under  a  Cl  format  than  under  an 
II  format.  Training  tasks  concerned  mainly  with  situation-specific,  sequential 
steps  (procedures),  on  the  other  hand,  appear  to  be  more  adequately  trained 
under  an  II  format  than  under  a  Cl  format.  This  suggestion  implies  that  any 
one  course  could  have  both  an  II  format  and  a  Cl  format  to  present  effectively 
different  kinds  of  training  content  to  students. 

In  summary,  the  results  concerned  with  the  interactive  effects  of  ability 
level  and  type  training  tasks  with  methods  of  instruction  suggest  that  no 
one  method  of  instruction  is  more  or  less  effective/efficient  overall.  Indeed, 
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some  combination  of  instructional  formats  within  a  given  course  may  promote 
training  effectiveness/efficiency  to  a  greater  extent  than  a  single  format. 
This  combination  of  formats  could  be  based  on  different  ability  levels  of 
incoming  students  and/or  different  kinds  of  generic  tasks  taught  in  the 
school. 

Table  38  presents  a  hypothetical  version  of  how  II  and  Cl  formats  may 
be  assigned  to  teach  different  amounts  of  generic  tasks  in  a  course  consider¬ 
ing  the  ability  level  of  students.  The  assignment  of  II  and  Cl  formats  is 
based  on  the  findings  in  the  present  study,  past  research  reviewed,  and  specu¬ 
lation.  The  purpose  of  presenting  this  table  is  to  suggest  one  approach 
that  instructional  designers  may  use  to  address  the  issue  of  different  formats 
within  one  course. 


TABLE  38.  INSTRUCTIONAL  FORMATS  HYPOTHESIZED  FOR  DIFFERENT  ABILITY  LEVELS 
OF  STUDENTS  AND  GENERIC  TRAINING  TASKS 


Training  Task  and  Amount  Taught* 

Fact 

Cateqory 

Procedure 

Rule 

Principle 

H 

L 

H  L 

H  L 

H  L 

H 

L 

High  Ability 
(mental  categories 

1  -  3U) 

I** 

I 

I  I 

I  I 

C***  I 

C 

C 

Low  Ability 
(mental  categories 
3L  -  4L) 

C 

C 

I  I 

I  C 

C  C 

C 

C 

*H  =  Relatively  high  amount;  L  =  Relatively  low  amount 
**I  =  Individualized  format 
***C  =  Conventional  format 


Note:  High  and  low  percentages  for  tasks  taught  are: 

Fact  and  category:  H:  16-30%,  L:  1-15% 

Procedure,  rule,  and  principle:  H:  51-100%;  L:  1-50% 


In  table  38,  under  fact,  it  was  assumed  that  higher  ability  students 
could  impose  their  own  structure  on  learning  facts,  but  that  lower  ability 
students  would  require  instructor  support  in  learning  large  numbers  of  facts. 
For  both  higher  and  lower  ability  students,  an  II  format  was  chosen  under 
category  tasks  because  it  was  assumed  that  sufficent  structure  was  inherent 
in  the  presentation  of  these  tasks  in  a  self-paced  mode  for  all  students  to 
learn.  An  II  format  was  chosen  for  higher  and  lower  ability  students  under 
high  amounts  of  procedure  tasks  taught,  and  was  based  on  the  task-specific 
TAS  findings  in  the  present  study.  A  conventional  format  was  assigned  to 
lower  ability  students  under  low  amounts  of  procedure  task,  based  on  the 
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task-specific  TAS  findings.  Higher  ability  students  were  assigned  an  II 
format  for  low  amounts  of  procedure  tasks  based  on  the  assumption  that  suffi¬ 
cient  task  structure  was  inherent  in  the  presentation  of  this  amount  of  proce¬ 
dure  tasks  in  a  self-paced  mode.  A  conventional  format  was  assigned  to  all 
students  under  high  amounts  of  rule  tasks  based  on  the  task-specific  TAS 
findings.  Low  amounts  of  rule  tasks  were  assigned  an  II  format  for  higher 
ability  students  based  on  task-specif ic  TAS  findings.  A  Cl  format  was  assigned 
for  lower  ability  students  based  on  the  assumption  that  lower  ability  students 
would  require  instructor  support  in  learning  the  generalization  aspect  of 
rule  tasks.  Conventional  formats  were  assigned  to  higher  and  lower  amounts 
of  principle  tasks  trained  to  all  students  based  on  the  task-specific  TAS 
findings. 

The  differences  between  II  and  Cl  formats  need  not  be  sharply  divided 
in  one  course  for  presenting  different  tasks.  A  conventional  format  could 
still  use  an  individualized  environment  (e.g.,  computer  terminal,  SP  carrel), 
but  could  provide  instruction  in  small  groups  via  peers  or  instructors.  Recent 
evidence  (Hopmeier,  1981)  has  suggested  that  screening  incoming  students  for 
personality  types  can  facilitate  performance  in  a  computer  assisted  instruc¬ 
tion  (CAI)  environment.  Modifying  an  II  context  to  incorporate  Cl  format 
changes  reduced  student  attrition  and  decreased  training  time  for  Navy  enlisted 
personnel.  These  changes  resulted  from  classifying  an  individual  as  intro¬ 
verted  (does  well  in  a  typical  II  format)  or  extroverted  (requires  the  "people" 
aspect  of  a  Cl  format).  In  addition,  as  noted  in  Vineberg  et  al.  (1971),  a 
significant  number  of  lower  mental  category  military  personnel  depended  more 
on  the  auditory  mode  rather  than  the  visual  mode  (i.e. ,  printed  material)  to 
obtain  job-related  information  than  higher  mental  category  personnel.  An 
issue  raised  by  these  findings,  then,  is  whether  lower  ability  students  may 
be  more  extroverted  when  learning,  and/or  whether  higher  ability  students 
tend  to  be  more  introverted  in  their  preference. 
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SECTION  V 

CONCLUSIONS  AND  RECOMMENDATIONS 

This  section  contains  conclusions  about  the  relative  effectiveness/ 
efficiency  of  II  for  different  kinds  of  training  tasks  and  ability  levels  of 
trainees.  Recommendations  for  II  in  Navy  technical  training  are  also 
presented. 

CONCLUSIONS 

Results  presented  in  this  report  support  the  following  conclusions  for 
the  courses  studied: 

1.  Individualized  instruction  (whether  it  is  self-paced  or  computer 
managed)  is  just  as  effective  as  conventional  instruction  in  preparing 
graduates  for  operational  jobs. 

2.  Individualized  instruction  benefits  higher  ability  students  during 
training  more  than  lower  ability  students. 

3.  Conventional  instruction  does  not  benefit  one  ability  group  more 
than  another  during  training. 

4.  Fleet  performance  ratings  do  not  reflect  school  performance  differ¬ 
ences  or  different  ability  levels  of  graduates. 

5.  When  course  content  is  classified  into  generic  training  tasks, 
fact  or  principle  tasks  are  more  difficult  and  time  consuming  for  students 
in  both  II  and  Cl  courses  than  category,  procedure,  or  rule  tasks.  Student 
end-of -course  grades  are  also  lower  in  courses  teaching  relatively  larger 
amounts  of  fact  or  principle  tasks. 

6.  When  the  TAS  items  rated  by  fleet  supervisors  were  classified  by 
generic  tasks  trained  in  both  II  and  Cl  courses,  II  was  found  to  be  more 
effective  than  Cl  when  courses  teach  primarily  procedure  tasks.  Conventional 
instruction  was  more  effective  than  II  when  courses  teach  primarily  rule  or 
principle  tasks. 

7.  No  one  method  of  instruction  is  more  or  less  effective/efficient 

in  training  different  types  of  training  tasks  to  students  of  different  ability 
levels.  Indeed,  some  combination  of  instructional  formats  within  a  given 
course  may  facilitate  training  effectiveness/efficiency  to  a  greater  extent 
than  a  single  format. 

8.  Definitive  conclusions  concerning  the  efficiency  (training  costs 
and  training  time)  of  II  for  Navy  training  could  not  be  reached  from  the 
data  of  this  study. 

9.  Geographic  location  of  the  school  and  sex  of  graduate  do  not  signi¬ 
ficantly  influence  overall  school  or  fleet  performance. 
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RECOMMENDATIONS 

Since  this  report  is  based  on  data  analyzed  across  schools,  no  specific 
recommendations  are  offered  to  improve  individualized  instruction  in  a  specific 
school.  The  following  recommendations  pertain  to  schools  grouped  by  instruc¬ 
tional  method  employed,  general  ability  rankings  of  students,  or  generic 
tasks  taught. 

Based  on  these  perspectives,  the  following  recommendations  are  offered: 

1.  Careful  consideration  should  be  given  to  any  attempt  to  individu¬ 
alize  indiscriminately  in  all  schools  in  the  Navy.  Rather,  the  feasibility 
of  employing  different  instructional  formats  (instructor  involvement,  self¬ 
pacing,  adaptive  sequencing,  computer  support)  with  differently  weighted 
contributions  in  a  given  course  should  be  investigated.  A  pilot  implementa¬ 
tion  and  an  ongoing  evaluation  of  a  chosen  alternative  could  then  be  initiated. 
The  goal  of  such  an  investigation  would  be  to  select  cost-effective  training 
systems  for  different  kinds  of  Navy  schools. 

2.  An  economic  analysis  of  subcomponent  categories  of  training  costs 
between  schools  could  reveal  resource  sources  contributing  different  amounts 
of  money  to  different  aspects  of  training.  Life  cycle  costs,  as  one  indicator 
of  long-term  training  efficiency,  could  be  used  in  conjunction  with  training 
effectiveness  measures  to  decide  among  training  efficiency/effectiveness  of 
alternative  systems. 

3.  Attrition  of  first-term  enlisted  personnel  is  an  issue  of  current 
interest.  Completion  of  first  tour  of  duty  could  be  used  as  a  measure  of 
attrition  and  could  be  collected  on  the  graduates  in  the  sample  selected  for 
this  study.  Analysis  of  attrition  data  would  provide  information  on  the 
relative  influence  of  the  technical  schools,  general  ability  level  of  the 
graduate,  method  of  instruction,  and/or  training  tasks  on  explaining  potential 
differences  in  attrition  rates  among  groups  of  graduates  in  the  sample  inves¬ 
tigated.  Training  effectiveness/efficiency  measures  at  the  school  level  and 

in  the  fleet  setting  could  also  be  treated  as  potential  indicators  of  attrition 

4.  More  objective  measures  of  training  effectiveness  need  to  be  iden¬ 
tified  and  collected  for  definitive  evaluations  of  alternative  training  systems 
Supervisor  ratings  are  subjective  measures  which  may  or  may  not  adequately 
discriminate  individual  capabilities  to  perform  on  a  job  for  which  training 
was  given.  Consideration  should  be  given  to  development  and  use  of  more 
objective  measures  such  as  job  knowledge  and  job  sample  tests  for  determining 
training  effectiveness.  The  relationships  between  supervisor  ratings  and 
these  types  of  tests  should  also  be  established. 
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SECTION  VI 
POSTNOTE 

i 

As  a  result  of  the  study,  needs  were  generated  for  additional  informa¬ 
tion  about  II.  Consequently,  follow-on  efforts  were  initiated.  One  effort 
is  concerned  with  SP  training  time  estimates.  Another  is  examining  practices 
used  in  conducting  SP  instruction.  These  efforts  were  initiated  in  part  by 
the  findings  of  Freda,  Ford,  &  Hall  (1981)  that  indicated  three  different 
data  bases  for  student  completion  times  in  individualized  courses  and  instruc¬ 
tor  preference  for  conventional  instruction  over  individualized  instruction. 

SP  TRAINING  TINE  ESTINATES 

While  the  research  literature  indicates  that  II  can  be  more  efficient 
than  Cl,  it  is  difficult  to  demonstrate  this  empirically  within  the  context 
of  a  "business  as  usual"  military  training  environment.  One  problem  is  that 
there  are  different  sources  of  information  concerning  student  completion 
times  in  Navy  self-paced  courses.  Currently,  the  TAEG  is  examining  several 
sources  of  information  to  determine  which  source  provides  the  most  reliable 
and  accurate  baseline  estimate  of  time-to-complete  training  under  SP  instruc¬ 
tion.  This  information  is  indispensable  to  determining  the  training  effici¬ 
ency  of  SP  instruction. 

PRACTICE  OF  SP  INSTRUCTION 

In  conducting  the  present  study,  the  realization  arose  that,  in  practice, 
SP  instruction  is  not  a  unitary  concept.  The  term  apparently  has  different 
meanings  to  different  individuals.  It  was  observed  during  visits  to  II  schools 
that  there  are  variations  in  the  ways  in  which  SP  instruction  occurs.  These 
variations  may  affect  what  II  can  achieve.  Thus,  TAEG  initiated  an  effort 
to  identify  the  variations  in  SP  instruction  and  to  assess  both  positive  and 
negative  factors  associated  with  the  different  self-pacing  strategies  as 
they  affect  training  efficiency  or  effectiveness.  The  thrust  here  is  not  to 
demonstrate  that  SP  instruction  is  an  efficient  training  method  (since  this 
seems  to  have  been  amply  demonstrated  by  previous  research)  but  rather  how 
best  to  achieve  the  efficiencies.  Thus,  this  study,  just  underway,  will 
develop  recommendations  for  optimizing  the  conduct  of  SP  instruction. 
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APPENDIX  A 

SlfMARY  OF  FINDINGS  ON  CAI  AND  CMI  COMPARED  TO  CONVENTIONAL  INSTRUCTION 

This  appendix  presents  the  Orlansky  and  String  (1979)  summary  of 
studies  on  the  use  of  II  in  military  settings. 
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APPENDIX  B 
ASVAB  INFORMATION 

This  appendix  describes  the  subtests  comprising  the  Armed  Services 
Vocational  Aptitude  Battery  (ASVAB).  It  also  provides  the  routines  for 
converting  ASVAB  scores  to  Armed  Forces  Qualification  Test  (AFQT) 
percentiles.  The  AFQT  percentiles  were  used  as  measures  of  general  ability 
for  the  present  TAE6  study.  The  information  presented  in  this  appendix 
pertains  to  ASVAB  usage  prior  to  October  1980.  The  TAEG  study  sample  of 
graduates  was  administered  the  pre-October  1980  ASVAB  version.  After 
October  1980,  several  changes  were  made  to  the  item  content  and  test  score 
interpretation  of  the  ASVAB. 
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TABLE  B-l .  ASVAB  SUBTESTS 


Subtest 

Abbrevia¬ 

tion 

Description 

leneral  Information 

GI 

A  15-item  general  knowledge  test,  primarily  on 
sports,  outdoor  activities,  automob i le  mechan¬ 
ics,  and  history.  Testing  time  is  7  minutes. 

Numerical  Operations 

NO 

A  50-item  speeded  mathematical  test,  requiring 
elementary  addition,  subtraction,  multiplica¬ 
tion,  and  division--3  minutes. 

Attention  to  Detail 

AD 

A  30-item  speeded  test  in  which  the  examinee 
counts  the  number  of  Cs  embedded  in  lines  of 

0s--5  minutes.  | 

rford  Knowledge 

WK 

A  30-item  vocabulary  test--10  minutes.  j 

Arithmetic  Reasoning 

AR 

A  20-item  arithmetic  test  requiring  examinees 
to  solve  word  problems--20  minutes. 

Space  Perception 

SP 

A  20-item  pictorial  test.  Requires  examinee 
to  select  the  three-dimensional  figure  that 
could  be  made  from  a  flat  pattern--12  minutes. 

Mathematics  Knowledge 

MK 

A  20-item  test  requiring  knowledge  of  algebra, 
geometry,  fractions,  decimals,  and  exponents  — 

20  minutes. 

Electronics  Information 

El 

A  30-item  test  requiring  knowledge  of  electrical 
and  electronic  components,  principles,  and 
symbols— 15  minutes 

Mechanical  Comprehension 

MC 

A  20-item  test  about  drawings  illustrating 
mechanical  principles--15  minutes. 

General  Science 

GS 

A  20-item  test  measuring  knowledge  in  the 
physical  (N  =  10)  and  biological  (N  =  10) 
sciences— 8  minutes. 

Shop  Information 

SI 

A  20-item  test  on  examinee's  knowledge  about 
the  use  of  shop  tools  and  practices—8  minutes. 

Automotive  Information 

AI 

A  20-item  test  on  automobile  parts,  operations, 
or  malfunctions—10  minutes. 

V'*  » • 

.«•••  as  Navy  Standard  Scores  (NSS)  having  a  mean  of  50  and  a 

*«•  •  i*v  or  of  10  fo*'  an  unrestricted  recruit  population. 


•?,  nfio^q  scores  for  WK,  AR  and  SP.  The  total  is  then 
V  '  !>*'■  »«t  '  ’>  -.nre, 

a  1 
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TABLE  B-2.  PRE-OCTOBER  1980  AFQT  CONVERSION:  ASVAB  FORMS  6  AND  7 


The  AFQT 

is  computed  by  adding  three  raw  component  scores 

:  WK+AR+SP.  The 

resulting 

total  raw  score 

is  then  converted 

to  an  AFQT  percentile  score  using 

the  following  conversion 

table. 

Total 

AFQT 

Total  AFQT 

Raw 

Percentile 

Raw  Percentile 

Score 

Score 

Score  Score 

70 

99 

40 

47 

69 

98 

39 

46 

68 

97 

38 

45 

67 

96 

37 

43 

66 

95 

36 

41 

65 

94 

35 

39 

64 

93 

34 

37 

33 

35 

63 

91 

32 

33 

62 

89 

31 

31 

61 

86 

60 

83 

30 

28 

59 

80 

29 

25 

58 

77 

28 

21  (Minimum  required 

57 

75 

for  enlistment) 

56 

73 

55 

71 

27 

19 

54 

69 

26 

16 

53 

67 

25 

13 

52 

65 

24 

11 

23 

10 

51 

64 

22 

8 

50 

62 

21 

7 

49 

60 

20 

6 

48 

58 

18-19 

5 

47 

56 

16-17 

4 

46 

55 

14-15 

3 

45 

54 

12-13 

2 

44 

52 

0-11 

1 

43 

50 

42 

49 

41 

48 

ASVAB  AFQT  Scores  Mental 

Cateqory 

93  to  99 

1 

65  to  92 

2 

49  to  64 

Upper 

3  (3U) 

31  to  48 

Lower 

3  (3L) 

21  to  30 

Upper 

4  (4U) 

10  to  20 

Lower 

4  (4L) 
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APPENDIX  C 


SME  TRAINING  PACKAGE 


This  appendix  is  a  copy  of  the  package  used  for  training  school  subject 
matter  experts  to  classify  course  learning  objectives  into  the  Instructional 
Quality  Inventory  (IQI)  system.  The  material  is  presented  in  4  parts  as 
follows: 

Part  I.  Classification  Procedures  for  SMEs  Based  on  a  Modified  Version 
of  the  IQI  System 

Part  II.  Question  Guide 

Part  III.  Practice  Items 

Part  IV.  Classification  Guide 

Material  contained  in  the  package  was  adapted  from  Ellis,  Wulfeck  and 
Fredericks  (1979). 
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PART  I.  CLASSIFICATION  PROCEDURES  FOR  SMEs  BASED 
ON  A  MODIFIED  VERSION  OF  THE  IQI  SYSTEM 


THE  INSTRUCTIONAL  QUALITY  INVENTORY  (IQI)  SYSTEM 

The  IQI  classification  scheme  is  shown  below.  Classification  of  the  TAS 
items  is  determined  according  to: 

1.  What  the  student  must  do;  that  is,  the  TASK  to  be  performed. 

2.  The  type  of  information  the  student  must  learn;  that  is,  instruc¬ 
tional  CONTENT. 

INSTRUCTIONAL  CONTENT 


REMEMBER 


USE 

UNAIDED 


USE  AIDE 


FACT  CATEGORY  PROCEDURE _  RULE _ PRINCIPLE 


tecall  or 

recognize 

names,  parts, 

dates,  places, 

vocabulary 

definitions, 

etc. 

Remember  the 
characteristics 
of  each  cate¬ 
gory  and  the 
guidelines  for 
classification. 

Remember  the 
steps  of  the 
procedure. 

Remember  the 
formula  or 
the  steps  of 
the  rule. 

Remember  the 
cause  and 
effect  rela¬ 
tionships  or 
the  state¬ 
ment  of  the 
principle. 

Classify  or 

Apply  the 

Apply  the 

Use  the 

categorize 

steps  of  the 

formula  or 

principle 

objects,  events. 

procedure  in 

rule  to  a 

to  inter- 

ideas,  accord- 

a  single  sit- 

variety  of 

pret  or 

ing  to  their 

uation  or  on 

problems  or 

predict  why 

characteristics. 

a  single 

situations , 

or  how 

with  no  memory 

piece  of 

with  no 

things 

aid. 

equipment, 
with  no 
memory  aid. 

memory  aid. 

happened 
or  wi 1 1 
nappen , 
with  no 

memory  aid.  | 

Given  category 

Given  steps  of 

Given  the 

Given  a  state-i 

characteris- 

the  procedure, 

formula  or 

ment  of  the  j 

tics  amd  guide- 

apply  the 

rule  steps, 

principle. 

lines,  cate- 

procedure  in 

apply  the 

interpret  j 

gorize  objects. 

a  single  sit- 

formula  or 

or  predict  t 

events,  ideas. 

uation,  or  on 

rule  to  a 

why  or  how  f 

according  to 

a  single  piece 

a  variety  of 

things  hap-  1 

characteris¬ 

tics. 

of  equipment. 

problems  or 
situations. 

pened  or  j 

will  happen. 

What  is  being  proposed  is  to  classify  each  TAS  item  into  one  of  the 
cells  in  the  above  matrix.  Inspect  the  matrix  briefly.  Note  the  five  types 
of  instructional  content  across  the  top,  and  the  three  types  of  task  levels 
down  the  side.  Each  cell  contains  a  general  description  of  the  performance 
task.  In  the  following  section  the  detailed  procedures  for  using  the  IQI 
are  discussed. 


CLASSIFICATION  PROCEDURE 

In  order  to  classify  each  TAS  item,  you  will  have  to  answer  only  4  questions. 
There  are  two  steps  in  the  classification  procedure.  The  first  step  determines 
what  the  student  must  do  to  perform  the  task  (task  level).  The  second  step 
determines  the  type  of  information  learned  to  perform  the  task  (content  type). 

Each  step  is  carried  out  by  answering  two  questions. 

Determining  the  Task  Level: 


Question  1 

Does  the  TAS  item  require  the  student  to 


Remember 


or 


Use 


information? 


The  remember-use  distinction  can  usually  be  made  by  looking  at  the  action 
verb  in  the  item.  Typical  action  verbs  are  listed  below.  The  ones  on  the  left 
usually  indicate  remember  tasks,  while  the  ones  on  the  right  usually  indicate 
use-level  tasks. 


REMEMBER 


USE 


name 

state  (from  memorv) 
list  (from  memory) 
recal 1 
remember 
rel ate 

write  (from  memory) 
recognize 

explain  (from  memory) 
describe  (from  memory) 


apply 

classify 

analyze 

derive 

demonstrate 

discriminate 

evaluate 

solve 

prove 

sort 

explain  (on  the 

maintain 

compute 

predict 

perform 

determine 


operate 

repair 

adjust 

calibrate 

remove 

replace 

assemble 

disassemble 

calculate 

basis  of  other  knowledge) 

troubleshoot 

load 

unload 
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Question  2 

If  the  student  uses  Information,  is  this  use  of  information 


Aided 


or 


Unaided 


? 


If  the  task  level  is  USE,  the  next  question  is  to  determine  whether  an 
AID  is  given.  This  can  be  done  by  looking  at  the  conditions  which  the 
TAS  item  is  to  be  performed.  Anything  that  replaces  the  need  for  MEMORY  counts 
as  an  aid.  If  the  student  must  perform  the  task  with  no  memory  aid,  then  the 
task  level  is  Use-Unaided. 


AIDS  include:  a  list  of  procedure  steps  from  a  tech. 

manual  or  MRC  card, 

a  formula  for  solving  problems, 

a  list  or  table  or  chart  of  characteristics, 

a  statement  of  a  principle. 

Normal  tools,  materials,  etc.,  are  NOT  aids. 


Determining  the  Content  Type 


Question  3 

Does  the  item  only  require  the  student  to  remember/use  information 
In  situations  or  on  types  of  equipment  which  were 
specifically  covered  or  previously  seen  in  training? 


Yes 


or 


The  most  Important  requirement  to  consider  is  whether  or  not  the  student 
will  have  to  deal  with  situations  or  types  of  equipment  not  previously 
encountered  during  training.  For  example,  if  all  major  instances  of  infor¬ 
mation  required  to  perform  the  TAS  item  were  not  presented  during  training, 
then  answer  No.  (Examples:  (a)  student  has  to  classify/identify  security 
codes  for  different  messages,  (b)  student  has  to  calculate  a  numerical  value 
using  a  mathematical  formula,  (c)  student  has  to  troubleshoot  a  piece  of  equip¬ 
ment  for  a  faulty  module.)  If  all  major  instances  were  presented  during 
training,  then  answer  Yes.  (Examples:  (a)  student  has  to  name  the  four  parts 
of  a  specific  piece  of  equipment,  (b)  student  has  to  perform  a  routine  mainte¬ 
nance  procedure  on  a  specific  piece  of  equipment.) 
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Question  4a 

If  you  answered  YES  to  question  3,  is  the  largest  percentage 
of  information  learned  to  perform  the  TAS  item  a 


FACT 


or 


PROCEDURE 


? 


FACTS.  Facts  are  what  you  think  they  are.  They  are  simple  associations 
between  objects,  events,  names,  parts,  functions,  locations,  dates,  etc. 

Facts  don't  have  to  come  in  pairs;  there  may  be  three  or  four  or  more  pieces 
of  information  that  go  together.  For  example,  a  student  might  have  to  remember 
the  name,  location,  and  function  of  each  of  the  parts  in  some  piece  of  equip¬ 
ment.  The  task  is  always  for  the  student  to  recall  them  or,  given  one  part 
of  the  fact,  to  recall  the  other  parts.  Facts  by  definition  must  be  presented 
during  training. 

Some  key  phrases  are: 

The  student  will  give  the  symbol  for  each  .... 

match  each  ...  with  its  .... 

1 i st  the  names  of  each  .... 

recal 1  the  dates  of  .... 

recall  the  location  and  function  of  each  .... 

given  the  ...  associated  with  each  - 

PROCEDURES.  A  procedure  is  a  sequence  of  steps,  performed  in  order,  on 
a  single  piece  of  equipment  or  In  a  single  situation.  At  the  REMEMBER  level, 
the  student  must  remember  the  steps  in  order.  At  the  USE  level,  the  student 
is  given  a  piece  of  equipment  or  a  situation,  and  must  perform  the  steps. 

The  job  requirements  for  procedures  Involve  single  pieces  of  equipment  or 
single  situations,  and  the  student  does  not  have  to  "generalize"  to  new 
equipments  or  sltualtons.  In  other  words,  everything  the  student  needs  to 
know  is  presented  during  training. 

Some  key  phrases  are: 

The  student  will  perform  the  steps  operating 

process  performing 

procedure  for  maintaining 

sequence  lighting  off 

etc. 
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Example: 

"The  student  will  field-strip  an  M-16  rifle." 


STEPS  OF 
PROCEDURE  FOR 
FIELD-STRIPPING 
AN  M-16  RIFLE 

(M-16  rifle) 

s, 

_ ^ 

(field-stripped  rifle) 

7 

Question  4b 

If  you  answered  NO  to  question  3,  is  the  largest  percentage 
of  information  learned  to  perform  the  TAS  item  a 


CATEGORY 

— 

or 

RULE 

or 

PROCEDURE 

CATEGORIES.  The  CATEGORY  content  type  is  used  when  the  job  requires  that 
a  large  number of  possible  objects,  events,  etc.,  be  classified  into,  or 
identified  as  a  member  of,  one  of  a  small  number  of  particular  categories. 
Instead  of  having  to  remember  each  object  and  its  classification,  the  student 
is  given  characteristics  for  each  category,  which  allow  him  to  classify 
objects,  etc.,  he  has  not  seen  before. 

Some  key  phrases  are: 


The  student  will  demonstrate  the 


characteristics 

features 

definition  of  each 
attributes 


type  of  . . . 
kind  of  . . . 
category  of 
classification 
situation 


; 


I 

i 


small 

number 

of 

possible 

categories 
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Example: 


"Given  a  series  of  sonar  scope  displays,  the  student  will  classify  them 
according  to  type  of  target." 


small 

number 

of 

possible 

types 


RULES.  A  rule,  like  a  procedure,  is  a  sequence  of  steps  and  decisions. 
However,  rules  can  be  applied  in  a  variety  of  situations  or  on  a  variety  of 
equipments,  whereas  a  procedure  is  applied  only  in  a  specific  situation  or  on 
a  specific  piece  of  equipment.  Instead  of  having  to  remember  each  problem  or 
go  through  the  steps  on  each  object,  the  student  Is  taught  a  RULE  which  allows 
him  to  deal  with  problems,  objects,  and  events  he  has  not  seen  before. 

Formulas  and  mathematical  calculations  always  involve  the  use  of  rules, 
unless  the  student  has  a  calculator  or  computer  that  does  it  for  him. 


Some  key  phrases  are: 


The  student  will  demonstrate  the 

formula 

rule 

law 

process 

steps 

solving 
deriving 
for  proving 

calculating 

determining 

etc. 

(problem  1) 

(problem  2) 

* 

[ 

(answer  to  problem  1) 
(answer  to  problem  2) 

s 

STEPS  OF 
THE  RULE 

> 

s 

# 

(large  or  Infinite 
number  of  possible 
problems) 

7 

or 

FORMULA 

A 

(large  or  infinite 
number  of  possible 
answers) 

Example: 

"Given  any  two  values  of  current,  voltage,  or  resistance  in  a  circuit, 
the  student  will  use  Ohm's  Law  to  solve  for  the  third  value." 
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PRINCIPLES.  Principles  involve  explanations  of  why/how  things  work,  or  do 

not  work  the  way  they  do  work,  or  predictions  about  "what  would  happen  if  _ " 

Principles  are  based  on  cause-effect  relationships,  theoretical  statements, 
statistical  associations,  or  physical  or  scientific  "laws."  Instead  of  having 
to  remember  each  possible  situation  or  event  and  its  effects,  the  student  is 
given  a  PRINCIPLE  which  summarizes  the  "how"  or  "why"  of  general  situations  or 
which  allows  the  student  to  predict  what  is  likely  to  occur  in  a  variety  of 
situations. 

Some  key  phrases  are: 

The  student  will  demonstrate  the  principle  of 

the  explanation  of 
how  ... 
why  . . . 

the  reasons  for 


Example: 


Based  on  his  knowledge  of  electronic  theory,  the  student  will  predict 
the  effect  In  the  circuit  shown  below  If  the  load  resistance,  or  the 
filter  capacitor,  were  shorted. 


(situation  requiring 
prediction  -  load 
resistance  shorted) 


(filter  capacitor 
shorted) 


^■‘electronic  theory 


— 3 


(prediction 
about  resulting 
circuit  behavior! 


(prediction 
about  resulting 
circuit  behavior 
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To  summarize,  there  are  five  content  types.  Two  content  types  involve 
performing  jobs  in  situations  or  on  equipment  specifically  encountered  during 
training.  Three  content  types  involve  dealing  with  equipment  or  situations  not 
specifically  presented  during  training.  This  explanation  is  presented  in  the 
following  table: 


All  Major  Instances 
Content  Type  Covered  in  Training? 


FACT 

Yes 

CATEGORY 

No 

PROCEDURE 

Yes 

RULE 

No 

PRINCIPLE 

No 

Sometimes  classification  can  be  confusing,  especially  between  FACTS  and 
CATEGORIES,  and  between  PROCEDURES  and  RULES.  The  way  to  resolve  problems  is  to 
"REMEMBER  THE  JOB";  that  is,  to  consider  carefully  what  the  student  must  be  able 
to  do  after  instruction. 

Again,  the  most  important  thing  to  consider  is  whether  the  student  will 
have  to  deal  with  objects  or  situations  that  he  has  not  seen  during  training. 

For  example,  if  the  student  were  required  to  sort  or  classify  things  according 
to  their  characteristics,  and  if  the  student  on  the  job  were  going  to  be  dealing 
with  things  not  seen  during  training,  then  the  content  would  be  a  CATEGORY. 
However,  suppose  instead  that  there  were  only  seven  objects  the  student  would 
ever  see.  Then,  it  would  be  more  efficient  to  teach  each  object  and  its  category 
name  as  a  FACT  (seven  facts  total). 

Similarly,  RULES  are  taught  so  that  the  student  can  apply  his  knowledge  to 
situations  he  did  not  encounter  in  training.  However,  suppose  the  situations 
are  so  similar  that  "if  you've  seen  one,  you've  seen  them  all."  This  would  be 
more  efficiently  taught  as  a  PROCEDURE. 

Example:  FACT  vs.  CATEGORY 

"Given  a  variety  of  metal  fasteners,  the  student  will  sort  them  according 
to  type  (bolts,  screws,  studs,  or  rivets)." 

This  could  be  taught  as  a  CATEGORY:  the  student  could  be  taught  the 
characteristics  of  bolts  (fine  threads,  blunt  end,  etc.),  the  characteris¬ 
tics  of  screws  (coarse  threads,  pointed  end,  etc.),  the  characteristics 
of  studs  (no  head,  fine  threads,  etc.),  and  the  characteristics  of  rivets 
(no  threads,  etc.).  However,  one  bolt  is  pretty  much  the  same  as  any  other 
bolt,  and  the  same  for  screws,  studs,  and  rivets,  except  that  they  come 
in  different  sizes.  Therefore,  it  might  be  more  efficient  to  teach  these 
as  four  FACTS:  bolt  -  appearance,  screw  -  appearance,  etc.  The  confusion 
here  can  be  solved  if  the  job  requirements  are  determined.  If  there  are 
lots  of  different  metal  fasteners,  and  the  student  will  see  new  bolts, 
etc.,  on  the  job,  then  the  content  type  is  CATEGORY.  If  there  are  only  a 
few,  and  they're  all  nearly  alike,  then  the  content  type  is  FACT. 
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Example:  PROCEDURE  vs.  RULE 


"Given  a  word  in  print,  correctly  spelled,  the  student  will  look  up  the 
word  in  a  dictionary,  and  state  its  definition  orally." 

This  might  appear  to  be  a  RULE:  There  are  a  large  number  of  possible 
words  (inputs),  and  a  large  number  of  possible  definitions  (outputs). 
However,  since  the  spelling  is  given,  its  easy  to  look  up  the  word: 

Find  the  first  letter  of  the  word,  find  that  chapter  in  the  dictionary, 
find  the  second  letter,  find  that  section  of  the  chapter,  etc.  This  is 
most  efficiently  taught  as  a  PROCEDURE. 

However,  suppose  the  word  was  given  orally  and  not  spelled.  This  would 
then  be  a  fairly  complicated  RULE,  involving  listening  skills,  phonemic 
translations,  etc. 
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PART  II.  QUESTION  GUIDE 

The  following  sequence  of  questions  should  be  used  to  assure  consistent 
judgments  in  classifying  each  TAS  item: 

1.  Does  the  TAS  item  require  the  student  to  remember  or  use  information? 

2.  If  USE  information,  is  performance  AIDED  or  UNAIDED? 

3.  Is  the  situation/equipment  in  the  item  specifically  covered  during  training? 

4a.  If  covered,  is  the  instructional  content  in  the  item  primarily  a  FACT  or 
PROCEDURE? 

4b.  If  not  covered,  is  the  instructional  content  in  the  item  primarily  a  CATEGORY, 
RULE,  or  PRINCIPLE? 


PART  III.  PRACTICE  ITEMS 


Use  the  preceding  question  guide  to  classify  the  following  sample  skill/ 
knowledge  items.  The  items  are  classified  according  to  the  scheme  presented 
earlier.  Compare  your  classification  with  the  one  given  for  each  item. 

Some  of  the  items  will  be  difficult  to  classify.  There  are  three  reasons 
for  this.  First,  many  of  them  are  not  “good"  ones;  they  are  written  in  such  a 
way  that  it  may  not  be  clear  what  behavior  is  required  or  what  content  is  to  be 
taught.  (They  are,  however,  fairly  typical.)  Second,  all  of  them  are  taken 
out  of  context  and  may  deal  with  unfamiliar  topics.  Therefore,  they  are  diffi¬ 
cult  to  classify,  because  information  about  the  job  is  not  provided.  Third,  some 
examples  were  chosen  deliberately  to  be  hard  to  classify,  so  that  classification 
problems  could  be  illustrated. 

For  the  reasons  given  above,  the  reader  should  net  expect  to  be  able  to 
classify  perfectly  these  objectives  and  test  items  (or  any  others)  immediately. 
Since  the  most  important  step  in  classification  is  REMEMBER  THE  JOB,  the  reader 
should  learn  to  ask  the  "right"  questions  of  job  experts,  so  that  bad  items  can 
be  revised,  and  so  that  unfamiliar  topic  areas  can  be  classified  reliably.  The 
examples  are  intended  to  illustrate  this  "question  asking"  process. 
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ITEM 


State  the  rule  for  finding  total  Inductance  in  a  series 
circuit. 


1. 

Task  Level? 

Content  Type? 


2. 

Task  Level? 

Content  Type? 


REMEMBER 


Does  the  student  have  to  remember  something,  or 
perform  a  task?  In  this  case,  the  student  simply  has 
to  recall  the  correct  rule  for  finding  total  inductance. 
Therefore,  the  task  level  is  Remember. 

RULE 


The  student  must  remember  a  rule  for  SOLVING  for 
total  inductance.  "SOLVE"  is  a  keyword  for  RULE. 

The  formula  for  total  Inductance  involves  a  series  of 
mathematical  operations  applicable  in  any  series 
circuit  having  inductors  with  various  possible  values. 
The  process,  then.  Is  a  series  of  steps  which  apply 
across  situations. 

It  Is  possible  to  confuse  a  piece  of  information 
taught  at  the  Remember  level  for  Categories,  Procedures, 
Rules,  or  Principles,  with  Remember-Fact  information. 

The  difference  is  that  Facts  cannot  be  used  immediately. 
In  this  case,  the  student  could  use  the  rule  to  solve 
inductance  problems. 

Given  pictures  of  the  collar  devices  for  different 
ranks  of  Naval  officers,  identify  the  ranks  they 
represent. 

REMEMBER 


The  task  here  is  "I  give  you  a  picture  of  a  collar 
device,  and  you  tell  me  the  rank."  Because  there  is 
a  limited  number  of  different  collar  devices,  the 
student  can  learn  each  device  and  its  name.  The  task 
is  Remember. 

FACT  _ 


The  content  type  is  Fact,  because  the  student  has  to 
memorize  pairs  of  symbols  and  names. 

Suppose  instead  that  there  were  a  large  number  of 
different  ranks  and  collar  devices,  but  that  different 
groups  of  them  had  similar  characteristics.  If  the 
student's  task  were  to  look  at  a  collar  device  and 
identify  which  group  it  belonged  to,  then  this  objective 
would  be  Use-Unaided  Category. 
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ITEM 

3.  Given  any  resistor  with  four  color  bands,  state  the  ohmic 

value  indicated  by  the  color  bands. 

Task  Level?  USE-UNAIDED _ 

If  there  were  only  a  limited  number  of  resistors,  the 
student  would  ever  have  to  deal  with,  he  could  memorize 
each  resistor's  colors  and  their  values  and  the  task 
level  would  be  Remember.  However,  there  are  many 
different  resistors  with  many  different  color  patterns. 
Therefore,  the  student  must  be  given  a  scheme  for 
determining  ohmic  value  from  the  color  pattern.  When 
the  student  applies  this  scheme  to  any  particular 
resistor,  he  is  USING  the  scheme.  If  he  has  no  memory 
aid,  then  the  task  level  is  Use-Unaided. 

Content  Type?  PROCEDURE _ 

In  order  to  make  the  content  type  decision,  we  need 
to  know  more  about  the  content  than  is  given  in  the 
objective.  In  particular,  we  need  to  know  what  the 
scheme  for  determining  ohmic  value  from  the  colors  is. 
This  is  a  good  time  to  consult  a  subject-matter  expert. 
It  turns  out  that  the  scheme  is  a  fairly  simple  sequence 
of  steps:  the  first  two  color  bands  indicate  the  first 
two  significant  figures  of  the  ohmic  value;  the  third 
band  is  the  number  of  zeros  to  add.  (The  fourth  band 
determines  tolerance,  not  value.)  If  the  scheme  were 
more  complicated,  and  involved  complex  calculations, 
the  content  type  would  be  Rule.  This  one  is  simple 
enough  to  be  a  Procedure. 

This  objective  should  not  be  confused  with  simply 
learning  the  meanings  of  the  colors.  Those  are  Facts 
which  the  student  mu,c  remember.  Those  facts  support 
this  Use-level  objective. 

4.  State  steps  involved  in  message  reception  and  duplicate 

checking,  as  listed  in  the  current  edition  of  NTP-4. 

Task  Level?  REMEMBER _ 

The  student  is  asked  to  recall  the  steps,  not  to 
do  anything  with  then. 

Content  Type?  PROCEDURE _ 

"Steps"  is  a  keyword  for  Procedure,  and  in  this  case 
the  task  described  in  the  test  item  appears  to  be 
procedural.  It  would  be  a  good  idea;  however,  to  check 
with  a  subject  matter  expert  to  make  sure. 
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5. 


Describe  the  principles  of  operation  of  a  rotary  gear  pump. 


Task  Level? 


Content  Type? 


6. 


Task  Level? 


Content  Type? 


7. 


Task  Level? 


Content  Type? 


REMEMBER  or  USE-UNAIDED 


It  is  difficult  to  determine  whether  this  objective 
is  Remember  or  Use-unaided  without  further  information. 
If  the  objective  had  said  "The  student  will  recall 
the  principles  then  it  would  clearly  be 

Remember.  On  the  other  hand,  if  the  objective  had 
said  "The  student  will  use  his  knowledge  of  hydraulic 
theory  to  explain  how  a  rotary  gear  pump  works,"  then 
it  would  be  Use-unaided.  It  is  necessary  to  determine 
what  the  student  must  do  after  training,  and  write  the 
objective  appropriately. 

PRINCIPLE 


This  objective  requires  the  student  to  explain  how 
or  why  the  pump  works.  The  content  type  is  Principle. 

Disassemble  and  reassemble  the  globe  valve  using  your  MRC 
job  program  card.  The  valve,  rags,  Prussian  blue,  gasket 
material,  packing,  and  tools  are  on  the  workbench. 

USE-AIDED 


The  MRC  job  program  card  gives  the  steps  to  be 
followed,  so  the  student  does  not  have  to  remember 
them. 

PROCEDURE 


This  is  a  series  of  steps  performed  on  a  single 
piece  of  equipment. 

For  each  of  the  call  signs  listed  below  identify  whether 
it  is  a  Navy  ship  call  sign,  a  Navy  shore  call  sign,  an 
indefinite  call  sign,  a  voice  call  sign,  a  task  organization 
call  sign,  or  not  a  valid  Navy  call  sign. 

_  USE-UNAIDED 


The  task  level  of  this  test  item  depends  on  complexity 
of  the  task.  If  there  were  just  a  few  call  signs,  then 
it  would  be  easy  to  memorize  their  types,  and  the  task 
would  be  Remember.  Here,  though,  it  turns  out  that 
there  are  several  thousand  call  signs,  and  the  student 
must  use  some  classification  scheme  to  identify  their 
type.  This  test  item  requires  the  student  to  use  the 
scheme  with  no  memory  aid. 

CATEGORY _ 


In  this  case  there  is  a  large  number  of  call  signs 
which  can  be  classified  into  a  small  number  of  cate¬ 
gories  or  types. 
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Ta^K  Level? 


Content  Type? 


Task  Level? 


Content  Type? 


Task  Level? 


Content  Type? 


Given  the  formula  for  Ohm's  Law,  and  two  values,  solve  for 
the  third  value. 

USE-AIOED 


The  student  must  use  Ohm's  Law  to  solve  for  a  value. 

In  this  case,  he  is  given  the  formula  as  a  memory  aid. 
He  does  not  have  to  remember  it.  The  task  level  is 
Use-aided. 

RULE 


"Solve"  is  a  keyword  for  Rules.  When  the  student  is 
asked  to  solve  something,  he  must  perform  a  series  of 
mathematical  operations  which  result  in  unique  solutions 
depending  on  the  values  used. 

Recall  the  duties  of  a  gun  captain. 

REMEMBER 


"Recall"  is  a  keyword  for  the  Remember  level.  The 
student  does  not  have  to  perform  the  duties,  only 
remember  what  they  are. 

FACT  or  PROCEDURE 


The  content  type  is  Procedure  if  the  student  is 
expected  to  remember  a  specific  sequence  of  steps 
which  the  gun  captain  must  perform.  However,  if  the 
student  is  to  recall  only  the  general  functions  of  a 
gun  captain,  the  objective  would  be  a  Fact.  Again,  it 
is  necessary  to  consult  a  subject-matter  expert  to 
make  this  decision. 

"File  30  drill  messages  representing  Top  Secret,  Secret, 
special  category,  readdressed  general  messages,  and  other 
classified  and  unclassified  messages  in  the  correct  files. 

USE-UNAIDED 


The  student  must  perform  the  task,  with  no  memory  aid. 
The  task  level  is  Use-unaided. 

CATEGORY  or  PROCEDURE 


If  the  student  must  inspect  each  message  and  determine 
category  membership  according  to  its  critical  charac¬ 
teristics,  then  the  content  type  is  Category.  If, 
however,  the  classification  of  the  message  is  obvious 
(e.g.  the  classification  Is  printed  on  the  top  line 
of  the  message),  then  this  is  a  procedural  task 
involving  filing.  Once  again,  the  services  of  a 
subject-matter  expert  are  required. 


Give  the  names  of  the  components  represented  by  the 
following  schematic  symbols... 


Task  Level? 


Content  Type? 


Task  Level? 


Content  Type? 


Task  Level? 


Content  Type? 


REMEMBER 


In  order  to  answer  this  test  item,  the  student  would 
have  had  to  memorize  the  names  of  the  schematic  symbols; 
therefore,  the  task  level  is  Remember. 

FACT 


The  key  phrase  in  this  test  item  is  "Give  the  names." 
Simple  associations  between  objects  or  symbols  and 
their  names  are  Facts. 

Given  the  explanation  of  the  principle  of  supply  of  a  mili¬ 
tary  force  from  the  text,  discuss  how  this  principle  applies 
in  Hitler's  attack  on  Russia,  the  Battle  of  Midway,  the 
Battle  of  the  Bulge,  and  Sherman’s  march  through  Georgia. 

USE-AIDED 


The  student  is  given  an  explanation  of  the  principle; 
therefore  the  task  is  Aided.  The  task  is  Use  because 
the  student  must  apply  the  principle  to  explain  specific 
battles.  It  is  assumed  that  these  battles  are  not 
discussed  in  the  text  and  have  not  been  described  in 
class.  If  they  had  been,  the  task  level  would  be 
Remember  and  there  would  be  no  reason  to  give  the 
student  an  Aid. 

PRINCIPLE 


The  student  is  given  a  principle  and  asked  to  interpret 
the  outcome  of  specific  battles.  The  key  words  here 
are  "principle"  and  "discuss." 

Recall  in  writing  the  lofargram  characteristics  for  the 
following  types  of  contacts:  merchant  ship,  aircraft 
carrier,  destroyer,  whale  diesel  submarine,  nuclear  submarine. 

REMEMBER 


The  student  does  not  have  to  do  anything  with  the 
characteristics  except  recall  them;  therefore,  the 
task  level  In  Remember. 

CATEGORY  or  FACT 


The  student  is  asked  to  memorize  characteristics  of 
several  categories  of  contacts  presumably  because 
he/she  will  later  have  to  look  at  lofargrams  and 
determine  what  the  contact  type  is.  Therefore,  the 


content  type  is  either  Fact  or  Category  depending 
on  whether  or  not  this  identification  requires 
generalization.  If  the  lofargrams  within  each  cate¬ 
gory  are  pretty  much  identical,  then  the  content  type 
would  be  Fact.  If  not,  then  the  content  type  is 
Category.  The  guidance  of  a  subject-matter  expert 
is  required. 

14.  Solve  for  inductive  reactance  In  a  circuit,  given  frequency 

and  Inductance. 

Task  Level?  USE-UNAIDED _ 

The  task  level  is  Use  because  the  student  is  required 
to  solve  problems.  It  is  not  aided  because  he/she 
is  not  given  the  formula. 

Content  Type?  RULE _ 

The  word  "solve"  is  a  keyword  for  Rule.  Here  the 
student  must  remember  the  formula  for  inductive 
reactance  and  then  use  it  to  solve  problems. 

15.  Perform  the  steps  required  to  accept,  verify,  and  log 

messages  to  be  transmitted  via  teletype  tape." 

Task  Level?  USE-UNAIDED _ 

The  student  must  "perform  the  steps,"  so  the  task  level 
is  Use.  The  student  has  no  memory  aid,  so  the  task 
is  Unaided. 

Content  Type?  PROCEDURE _ 

The  word  "steps"  is  a  keyword  for  Procedure  or  Rule. 

If  accepting,  verifying,  and  logging  is  complicated 
enough  so  that  doing  one  or  two  does  not  mean  that  the 
student  could  do  any  new  message,  then  the  content  type 
would  be  Rule.  In  this  case,  though,  the  steps  are  the 
same  for  every  message,  so  the  content  type  is  Procedure 

16.  State  the  principles  of  operation  of  a  jet  pump  as  described 

in  Propulsion  Engineering  Mod  6,  Lesson  2. 

Task  Level?  REMEMBER _ 

The  student  simply  has  to  recall  how  a  jet  pump  works, 
as  described  In  his  training  manual. 

Content  Type?  PRINCIPLE _ 

The  student  is  asked  to  remember  "how"  something  works. 
Explanations  of  how  things  work  are  Principles. 
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17. 


Use  the  principle  of  electromagnetic  induction  to  describe 
the  operation  of  an  AC  generator. 


Task  Level? 


USE-UNAIDED  or  REMEMBER 


The  task  level  depends  on  what  the  student  will  be 
taught  during  the  course;  that  is,  on  what  the  other 
objectives  are.  If  the  student  must  use  his  knowledge 
of  electromagnetic  induction  to  describe  something  not 
yet  taught,  then  the  task  is  Use-unaided.  If  the 
student  had  been  taught  the  principles  of  operation 
of  an  AC  generator  on  the  basis  of  electromagnetic 
induction,  then  all  the  student  would  have  to  do  is 
Remember. 


Content  Type?  PRINCIPLE _ 

The  student  must  explain  "how"  an  AC  generator  works. 

18.  Given  the  guidelines  for  determining  message  security 

classification,  determine  the  security  classification  (Top 
Secret,  Secret,  Confidential,  or  Unclassified)  for  outgoing 
messages. 

Task  Level?  USE-AIDED _ 


The  student  must  perform  a  task:  determine  appropriate 
security  classifications  for  a  variety  of  messages. 

The  task  level  is  Use.  To  the  extent  that  the  given 
guidelines  provide  sufficient  information  to  determine 
classifications  for  varied  messages,  the  task  is  Aided. 


Content  Type?  CATEGORY 


It  is  expected  that  the  student  could  take  any  message 
and  determine  which  of  the  four  security  classifications 
it  belongs  in.  On  the  job,  he  will  be  required  to  apply 
the  guidelines  to  messages  he  has  not  seen  before. 
Presumably  the  messages  will  be  varied  enough  so  that 
doing  one  is  not  like  doing  them  all.  The  given  guide¬ 
lines  should  contain  characteristics  of  messages  that 
help  determine  the  type  of  classification. 


From  the  examples  above,  it  should  be  clear  that  the  key  to  classification 
is  REMEMBER  THE  JOB.  Whenever  there  is  doubt  about  the  classification  of  an 
item,  a  subject-matter  expert  or  technical  manual  should  be  consulted,  so  that 
information  about  the  job  can  be  obtained. 
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PART  IV.  CLASSIFICATION  GUIDE 

Table  C-l  presents  alphabetically  some  use-level  action  verbs  classified 
by  instructional  content.  The  checkmarks  represent  typical  classifications  of 
these  verbs  by  Information  content.  Note  that  these  classifications  could 
change  due  to:  (1)  the  content  of  the  skill  item  from  which  these  verbs 
were  taken  or  (2)  relevant  expert  knowledge  provided  by  a  subject  matter  expert 
(SME).  Thus,  use  this  table  only  as  a  guide  when  classifying  TAS  items. 


TABLE  C-l .  ACTION  VERB  MATRIX 


INSTRUCTIONAL 

CONTENT 

USE-LEVEL 
i  ACTION  VERBS 

FACT 

CATEGORY 

PROCEDURE  RULE 

PRINCIPLE 

ADJUST 


ANALYZE 


APPLY 


ASSEMBLE 


CALCULATE 


CALIBRATE 


CATEGORIZE 


CHOOSE 


CLASSIFY 


COMPUTE 


DERIVE 


DETERMINE 


DIAGNOSE 


DISASSEMBLE 


DISCRIMINATE 


EXPLAIN 


EVALUATE 


IDENTIFY 


LIST 


LOAD 


MAINTAIN 


MATCH 


NAME 


OPERATE 


PERFORM 


PREDICT 


PREPARE 


PROVE 


RECOGNIZE 


REMOVE 
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TABLE  C-l.  ACTION  VERB  MATRIX  (continued) 


USE-LEVEL 

ACTION  VERBS 

INSTRUCTIONAL 

CONTENT 

FACT 

CATEGORY 

PROCEDURE 

RULE 

PRINCIPLE 

REPAIR 

X 

REPLACE 

X 

SELECT 

X 

SOLVE 

X 

SORT 

X 

TROUBLESHOOT 

X 

UNLOAD 

X 
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APPENDIX  D 
COST  DATA 


This  appendix  describes  how  training  cost  data  were  computed  for  use  in 
the  present  study.  A  strategy  for  choosing  among  alternate  training  systems 
on  cost  and  effectiveness  bases  is  also  described. 

Training  costs  should  include  both  complete  investment  and  operating 
costs.  Investment  costs  are  "front-end"  expenditures  associated  with  such 
items  as  equipment,  classroom  buildings,  curriculuum  development.  Operating 
costs  are  "ongoing,"  repetitive  expenditures.  They  include  items  such  as 
staff  costs  (e.g.,  pay,  health,  food),  student  costs  (pay,  health,  food, 
time  in  school,  rate),  and  travel. 

In  the  Per  Capita  Course  Costing  Data  Base  maintained  by  the  Chief  of 
Naval  Education  and  Training  (CNET),  many  investment  costs,  such  as 
facilities  and  curriculum  development  costs,  are  not  included.  The  data 
base  emphasizes  operating  costs  which  are  primarily  military  pay  and 
allowances  (MPN)  and  operating  and  maintenance.  Navy  (O&MN)  costs.  MPN 
reflects  military  staff  and  student  costs:  O&MN  costs  account  for  civilian 
pay,  some  supplies,  and  travel.  An  "other"  category  is  also  used  to 
classify  miscellaneous  items.  Based  on  the  cost  data  available  for  the 
schools  in  the  present  TAEG  study,  MPN  accounted  for  approximately  74 
percent,  O&MN  for  approximately  26  percent,  and  "other"  for  less  than  1 
percent  of  total  training  costs. 

The  incremental  costing  model  (System  I)*  was  used  with  the  CNET  Per 
Capita  Course  Costing  Data  Base  in  the  present  TAEG  study  to  calculate  the 
total  training  costs  to  produce  one  graduate  per  average  course  session 
(based  on  FY-79  dollars).  Costs  to  produce  an  individual  graduate  per 
specific  course  were  computed  by  multiplying  the  average  course  cost  to 
produce  one  graduate,  by  the  ratio  of  a  given  student's  time  to  complete  the 
course  to  the  average  course  completion  time.  This  metric  provided  the 
variance  required  to  use  training  costs  as  a  criterion  variable  (see  section 
III  in  the  report  concerning  training  costs  for  application  of  the  metric  in 
the  analyses).  However,  since  the  full  measure  of  resource  and  development 
costs^  are  not  adequately  stated  in  the  data  base,  the  results  of  training 
cost  analyses  should  be  viewed  with  caution. 


lw.  M.  Swope,  Cynthia  Yelvington,  and  J.  M.  Corey,  Incrementa'  Costing 
Model  for  Use  with  the  CNET  Per_Capita  Course  Costing  Data  Base:  System  I. 
TAEG  Report  No.  77,  November  1979.  Training  Analysis  and  Evlauation  Group, 
Orlando,  FL  32813  (A0AD81759). 

2j.  M.  Corey,  "The  full  measure  of  resource  costs."  Defense  Management 
Journal.  Third  Quarter,  1980,  18-23. 
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The  selection  of  appropriate  variables  and  measures  is  important  not 
only  for  training  costs  but  also  for  training  effectiveness.  Figure  D-l 
presents  a  theoretical  outcome  matrix  of  possible  decisions  based  on  cost 
and  effectiveness  factors  underlying  the  alternative  systems/programs  being 
compared.  Full  measures  of  costs  and  effectiveness  are  required  to  diminish 
the  occurrence  of  dilemmas  in  choosing  a  cost-effective  alternative. 


COST 


EFFECTIVENESS 


LOWER 

SAME 

HIGHER 

HIGHER 

NO 

NO 

? 

SAME 

NO 

EQUAL 

YES 

LOWER 

? 

YES 

YES 

Figure  D-l.  Decision  Outcome  Matrix  of 

Cost  and  Effectiveness  Comparisons 


•/ 

The  Decision  Outcome  Matrix  shown  considers  two  factors:  cost  and 
effectiveness.  Each  factor  is  gradated  ordinally.  Each  matrix  cell 
represents  the  joint  outcome  of  a  specified  alternative  system/program  being 
compared  to  another  on  cost  and  effectiveness  factors.  It  is  assumed  that 
decision  outcomes  can  be  obtained  from  at  least  an  ordinal  scale  of 
measurement. 

Note  that  out  of  the  possible  9  outcomes,  the  two  marked  by  a  "?" 
suggest  other  areas  of  possible  concern  in  cost  and  effectiveness 
comparisons.  More  factors  may  be  taken  into  consideration  when  a  highly 
effective  system  is  the  most  costly  alternative,  or  when  a  less  effective 
system  is  the  least  costly.  The  joint  outcome  may  be  expressed  as  the 
desirability  and/or  certainty  of  selecting  a  specified  alternative 
system/program.  For  example,  system  A  costs  less  than,  and  is  more 
effective  than,  system  B.  Thus,  a  desirable  outcome  of  YES  is  indicated  in 
the  matrix  for  selecting  A.  However,  if  A  costs  more,  as  well  as  being  more 
effective  than  B,  then  a  "?"  desirability  is  indicated. 

One  strategy,  to  prepare  for  all  possible  outcomes,  is  to  use  a  multi¬ 
variable.  mul ti -measurement  approach  to  determine  cost  and  effectiveness, 
the  multi -variable  aspect  of  this  approach  refers  to  an  expansive  effort  to 
select  those  classes  of  measures  that  contribute  totally  or  proportionately 
to  the  cost  and/or  to  the  effectiveness  of  the  comparison  systems/programs. 
Then,  once  the  variables  are  selected,  various  measures  can  be  employed  to 
represent  the  same  or  different  variables.  This  multi -measurement  aspect 
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increases  the  probability  of  observing  a  correlated  pattern  among  the 
measures  for  all  the  variables  used  to  reflect  cost  and  effectiveness. 
Further  analyses  are  conducted  to  determine  the  significance  of  obtained 
correlated  patterns.  The  multi-variable,  multi -measurement  approach  may 
also  lessen  the  occurrence  of  "?"  outcomes,  if  such  outcomes  are 
predominately  Influenced  by  an  insufficient  number  of  variables  and/or 
measures  selected. 
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APPENDIX  E 

COURSES  INCLUDED  IN  THE  STUDY 

The  full  names  and  locations  of  the  schools/courses  examined  during  the 
study  program  are  presented  in  this  appendix. 
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TABLE  E-l.  COURSES  INCLUDED  IN  THE  STUDY 


School /Course  Name 

Symbol 

Location 

Method  of 
Instruction 

Hull  Maintenance  Technician 

HT-II 

San  Diego,  CA 

Mixed* 

Class  "A"  School  Phase  II 

Radioman  "A"  Sea  School 

RM-SEA 

San  Diego,  CA 

GP2 

Radioman  "A"  Shore  School 

RM-SHORE 

San  Diego,  CA 

GP2 

Damage  Control  Assistant  "A2"  Course 

DCA 

Treasure  Island, 

CA 

GP3 

Hull  Maintenance  Technician 

Class  "A"  School  Phase  I 

HT-1 

Treasure  Island, 

CA 

GP 

Damage  Control  Assistant  "A2"  Course 

DCA 

Philadelphia,  PA 

SP3 

Hull  Maintenance  Technician 

Class  "A"  School  Phase  I 

HT-1 

Philadelphia,  PA 

GP 

Hull  Maintenance  Technician 

HT-II 

Philadelphia,  PA 

SP 

Class  "A”  School  Phase  II 

Electrician's  Mate  "A"  School 

EM 

Great  Lakes,  IL 

GP 

Engineman  "A"  School 

EN 

Great  Lakes,  IL 

4 

CMIH 

rire  Control  Technician 

"A"  School  Phase  I 

FT 

Great  Lakes,  IL 

GP 

Gunner's  Mate  "A"  School 

GM 

Great  Lakes,  IL 

GP 

Hach ini st's  Mate  600  psi  "A"  School 

MM-600 

Great  Lakes,  IL 

SP5’6 

lachinist's  Mate  1200  psi  "A"  School  MM-1200  Great  Lakes,  IL 
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TABLE  E-l.  COURSES  INCLUDED  IN  THE  STUDY  (continued) 


School /Course  Name 

Symbol 

Location 

Method  of 
Instruction 

Instrumentman  "A"  School 

IM 

Great  Lakes,  IL 

SP 

Aviation  Support  Equipment 

Technician  Electrical  "Al"  Course 

ASE 

Millington,  TN 

GP 

Aviation  Machinist's  Mate  "A"  School 

AD 

Millington,  TN 

CMI7 

Aviation  Electrician's  Mate  "A"  School 

AW 

Millington,  TN 

GP 

Aviation  Technician  Training 

Deviceman  "A"  School 

TD 

Millington,  TN 

SP 

’ersonnelman  "A"  School 

PN 

Meridian,  MS 

SP 

Yeoman  "A"  School 

YN 

Meridian,  MS 

SP 

lisbursing  Clerk  "A"  School 

DK 

Meridian,  MS 

SP 

Aviation  Storekeeper  "A"  School 

AK 

Meridian,  MS 

SP 

Ship's  Serviceman  "A"  School 

SH 

Meridian,  MS 

GP 

^Group-assisted  self-paced  (GRASP) 

^CMI  data  from  RM  Basics  obtained  from  CNTECHTRA  (CDP  6144;  894  records) 

3 

Officer  course 

^CMI  data  from  PE  Basics  obtained  from  CNTECHTRA  (CDP  6261;  359  records) 

5CMI  data  from  PE  Basics  obtained  from  CNTECHTRA  (CDP  6262;  552  records) 

®1978  student  training  records  obtained  from  National  Archives  via  CNTECHTRA 
and  PE  School. 

7CMI  data  obtained  from  CNTECHTRA. 
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APPENDIX  F 

DISCUSSION  OF  THE  PARTIAL  HIERARCHICAL  REGRESSION  MODEL 

The  partial  hierarchical  regression  model  used  in  data  analyses  is 
discussed  in  this  appendix.  Terms  used  in  the  model  are  explained  and  the 
model  is  specified  in  mathematical  notation. 

DESCRIPTION  OF  THE  PARTIAL  HIERARCHICAL  REGRESSION  MODEL 

A  partial  hierarchical  regression  model  was  used  to  analyze  the  data  of 
the  study.  A  hierarchial  model  is  one  in  which  the  predictors  are  entered 
cumulatively  into  the  regression  equation  according  to  a  specified  hierarchy 
(or  order)  dictated  in  advance  by  the  purpose  and  logic  of  the  study.  In 
the  present  TAEG  study,  this  hierarchy  was  directed  by  the  tasking 
questions;  method  of  instruction  was  entered  first;  ability  level  and 
training  tasks  second;  and  the  two-way  and  three-way  interactions  formed  by 
the  previously  entered  predictors  were  entered  subsequently.  Since  the 
predictors  were  entered  as  groups  (sets)  in  each  step  of  the  equation, 
rather  than  entered  individually,  the  model  is  called  partial  hierarchical 
regression.  In  partial  hierarchical  regression,  the  variables  within  each 
group  (or  set)  are  analyzed  in  a  forward  stepwise  manner^  at  each  step  of 
the  equation,  and  each  group  of  variables  is  analyzed  hierarchically  between 
sets.  The  R2  for  all  predictor  sets  can  thus  be  analyzed  into  increments  in 
the  proportion  of  the  criterion  variance  due  to  the  addition  of  each  new  set 
of  predictors  to  those  higher  in  the  hierarchy.  These  increments  in  R2  are 
squared  multiple  semipartial  correlation  coefficients  and  represent  the 
Increments  of  criterion  variance  accounted  for  by  a  given  set  of  predictors 
beyond  what  has  already  been  accounted  for  by  the  prior  sets.  An  advantage 
of  the  hierarchical  model  is  that  once  the  order  of  the  predictor  sets  has 
been  specified,  a  unique  partitioning  of  the  total  criterion  variance 
accounted  for  by  all  predictor  sets  may  be  made. 

An  issue  raised  by  the  use  of  the  partial  hierarchical  regression  model 
concerns  the  basis  for  determining  the  extent  of  contribution  of  any  group 
of  variables  once  the  entry  order  is  specified  in  the  equation.  Since  the 
contribution  of  the  first  set  of  predictors  is  determined  from  its  initial 
entry  in  the  equation,  subsequent  potential  changes  in  the  contribution  by 
the  first  predictor  set  when  other  variables  are  entered  later  into  the 
equation  do  not  influence  the  decision  made  from  the  initial  entry  of  the 
first  predictor  set.  For  example,  if  the  standardized  partial  regression 
coefficient  of  the  first  predictor  set  moves  toward  zero  when  later 
predictor  sets  are  added  to  the  equation,  the  situation  is  one  of  simple 
redundancy  between  the  first  and  later  predictor  sets  in  accounting  for  the 
criterion  variance.  If,  on  the  other  hand,  the  regression  coefficients  of 
the  first  predictor  set  changes  sign  or  increases,  the  relationship  between 
the  first  and  later  predictor  sets  Is  one  of  suppression  (see  Cohen  &  Cohen 
(1975)  for  further  discussion  of  this  issue).  With  respect  to  the  present 
TAEG  study,  examination  of  the  standardized  partial  regression  coefficients 

^Forward  stepwise  regression  selects  from  a  group  of  predictors  the  one 
predictor  at  each  stage  that  has  the  largest  squared  semipartial 
correlation  coefficient,  and  hence  makes  the  largest  contribution  to  R2. 
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of  the  first  predictor  set  of  methods  of  instruction  revealed  no  significant 
or  relevant  changes  when  the  predictor  set  of  ability  level  and  training 
tasks  was  entered  later  in  the  equation.  Thus,  entering  methods  of 
instruction  into  the  equation  first  (based  on  the  tasking  questions)  did  not 
result  in  redundancy  or  suppressive  effects  not  explained  by  main  effects  or 
interactions. 

EXPLANATION  OF  REGRESSION  TERMS  COMMON  TO  ALL  EQUATIONS  IN  THE  MODEL 

r2  refers  to  the  proportion  of  the  criterion  variance  shared  with  the 
optimally  weighted  predictor  variables. 

r2 Increment  is  the  cumulative  increase  in  the  proportion  of  criterion 
variance  explained  due  to  the  addition  of  each  set  of  predictors  entering  at 
the  point  in  the  equation,  beyond  the  criterion  variance  already  accounted 
for  by  the  previously  entered  a*ts  of  predictors.  These  increments  in  R? 
are  squared  semipartial  correlation  coefficients.  They  represent  the  unique 
contribution  of  a  predictor  to  R2  in  the  context  of  the  remaining  predictors 
(expressed  as  a  proportion  of  the  total  criterion  variance,  since  the 
effects  of  the  remaining  predictors  have  been  removed  from  the  specific 
predictor  but  not  from  the  criterion). 

B  refers  to  the  raw  score  partial  regression  coefficient.  B  is  the 
constant  weight  by  which  each  value  of  a  predictor  is  to  be  multiplied  in 
the  equation  and  reflects  the  average  or  expected  change  in  the  criterion 
value  for  each  unit  increase  in  each  predictor  when  the  values  of  the 
remaining  predictors  are  held  constant. 

C  is  the  regression  constant  or  Y  intercept  for  each  equation,  and 
serves  to  make  appropriate  adjustments  for  differences  between  the  mean 
predictor  values  and  mean  criterion  values  (e.g.,  gives  the  predicted 
criterion  value  when  predictor  values=0;  i.e.,  is  the  mean  of  the  criterion 
variable). 

A 

X  refers  to  the  predicted  value  of  the  criterion  variable.  In  the 
regression  model,  Y  represents  predicted  values  for  each  of  the  criterion 
variables  used  in  the  present  study. 

X  refers,  in  general,  to  a  set  of  predictor  variables. 
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SPECIFICATION  OF  THE  MODEL 


The  number  preceding  each  equation  refers  to  the  order  (step)  in  which 
each  set  of  predictors  was  entered  into  the  regression  equation. 

1.  =  V’B-jX-j  +  C 

c_.i=l,2 


where  X-j  is  composed  of  two  coded  predictors  each 
representing  different  contrasts  between  methods  of 
instruction. 


where  Xj  is  composed  of  six  continuous  predictors, 
one  representing  AFQT  and  the  remaining,  five 
types  of  training  tasks  in  percentages  of  task 
taught  in  school. 


4. 


where  X|<  is  composed  of  12  two-way  interaction  terms, 
10  of  which  were  formed  from  two  predictors  of  methods 
of  instruction  and  five  predictors  of  training  tasks, 
and  two  from  two  predictors  of  methods  of  instruction 
and  one  predictor  of  ability  level. 


c 


5. 


where  X]  is  composed  of  five  two-way  terms  that  were 
formed  from  one  predictor  of  ability  level  and  five 
predictors  of  training  tasks. 


Bk*k  + 
k*l,12 


y  Bixi  +y  Bmxm  + 

L  1-1,5  Z_rn=l,10 


C 


6a. 


where  X™  is  composed  of  10  three-way  interaction  terms 
formed  from  two  predictors  of  methods  of  instruction, 
one  predictor  of  ability  level,  and  five  predictors  of 
training  tasks. 


B|T#m  + 
m»l,10 


BS  +  C 


where  S  Is  one  coded  predictor  representing  sex  of 
graduate. 
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6b. 


A 
Y  = 


S’*!? 


m=l,  10£_,  n=l,3 


6c. 


A 

Y 


where  Ln  is  composed  of  three  coded  predictors 
representing  school  locations. 


E?ii  kiiME  ?li]s+Z^iTw 


BG  + 


C 


6d. 


A 

Y 


BiXi 

1«1,2 


where  G  is  one  continuous  predictor  representing 
end-of -course  grades.  G  was  used  as  a  predictor 
only  when  Y  =  TAS  rating. 


Zj-1.6  Z-k«l,12Z-l-l,5  Z-m=l,10 


BT  + 


where  T  is  one  continuous  predictor  representing 
training  time.  T  was  used  as  a  predictor  only 
when  Y  =  TAS  rating  or  training  cost. 


Predictor  sets  and  individual  predictor  variables  are  delineated  below. 


Xi  =  Tl,  T2  where  T1  =  II  (SP  +  CMI)  vs  Cl;  contrast  coded  as  II  (SP  = 

CM I  =  -H)  =  -1,  Cl  =  1,  and  T2  =  SP  vs  CMI,  coded  SP  =  -1, 
CMI  ■  1,  Cl  ■  0. 

Xj  =  A,  II,  12,  13,  14,  15  where 

A  =  AFQT  Percentile 

11  =  Percentage  of  Fact  Task 

12  =  Percentage  of  Category  Task 

13  =  Percentage  of  Procedure  Task 

14  =  Percentage  of  Rule  Task 

15  =  Percentage  of  Principle  Task 

*k  =  Till,  T1I2,  T1I3,  T1I4,  T1I5,  T2U, 

T2l2.  T2I3,  T2I4,  T2I5,  ATi,  AT2 

X]  =  All,  AI2,  AI3,  AI4,  AI5 

*m  »  ATiU,  AT1I2,  AT!l3,  Aim,  AT x 1 5 , 

AT2I1,  AT2I2,  AT2I3,  AT2I4,  AT2I5 

S  =  Sex  of  graduate;  dummy-variable  coded  as  male  =  1,  female  =  0 

Ln  =  LI,  L2,  L3;  effects  coded  as 

LI  *  1  if  San  Diego,  CA 
and  -1  if  Meridian,  MS 
L2  =  1  if  Great  Lakes,  IL 
and  -1  if  Meridian,  MS 
L3  =  1  if  Millington,  TN 
and  -1  if  Meridian,  MS 
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G  *  Average  end-of -course  grades  (percent  correct) 

T  *  Training  time  (contact  hours) 

The  increments  in  R2  accounted  for  by  the  previously  mentioned  predictor 
sets  can  be  described  by  the  following  equation: 

R2Y.XiX2X3...k  =  R2YXi  +  r2Y(X2.Xi)  +  R2Y(X3.XiX2)  +  •••  +  R2Y(k . X1X2X3 . . .k  -1) 

The  increments  in  R2  are  squared  multiple  semipartial  correlation 
coefficients.  Each  R2  increment  is  an  increase  to  the  proportion  of  criterion 
variance  accounted  for  by  a  given  predictor  set  beyond  what  has  already  been 
accounted  for  by  prior  sets  (i.e.,  sets  higher  up  in  the  hierarchy). 

Further,  the  amount  of  the  increment  in  criterion  variance  accounted  for  by 
that  set  can  not  be  influenced  by  criterion  variance  associated  with 
subsequent  sets  (i.e.,  those  lower  in  the  hierarchy). 
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APPENDIX  6 

MISSING  DATA  ANALYSIS 

This  appendix  presents  the  findings  (with  accompanying  statistical 
tables  and  breakdown)  of  the  missing  data  analysis.  The  purpose  of  this 
analysis  was  to  determine  if  graduates  with  data  present  differed  from  those 
with  data  absent.  Headings  used  in  this  subsection  are  listed  by  the 
criterion  variables. 

CRITERION:  PRESENCE  VS.  ABSENCE  OF  TAS  RATINGS 

Only  main  effects  (and  no  interactions)  were  found  in  the  regression 
analyses  presented  in  table  G-1. 


TABLE  G-1.  SIGNIFICANT  RESULTS  FROM  REGRESSION  ANALYSES  OF 
ABSENT  TAS  RATINGS 


Step 

Variables 

B 

F 

DF 

R2 

Increment 

Overall  F_  DF 

1 

II  vs  Cl  (Tl) 

-.16 

234.46 

1/4003 

.0811 

176.54  2/4003 

SP  vs  CMI  (T2) 

.13 

184.89 

1/4003 

2 

AFQT  (A) 

.002 

18.25 

1/3998 

.0375 

34.03  4/3998 

Fact  (11) 

.02 

56.02 

1/3998 

Principle  (14) 

-.003 

40.88 

1/3998 

The  following  findings  are  listed  by  significant  predictor  variables. 

1.  Method  of  Instruction.  A  larger  percentage  of  TAS  ratings  was 
absent  from  both  SP  and  CMI  graduates  than  from  GP  graduates.  TAS  ratings 
were  absent  more  from  CMI  graduates  than  from  SP  graduates  (see  table 
G-2). 


TABLE  G-2.  ABSENT  TAS  RATINGS  BY  METHOD  OF  INSTRUCTION 


Original 

Sample 

Size 

Absent 

TAS  Ratings 

N 

N 

X 

SP 

1451 

560 

39 

CMI 

823 

536 

65 

GP 

1732 

495 

29 

TOTAL 

4006 

1591 

40 
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2.  Ability  Level .  TAS  ratings  were  absent  more  from  data  of 
graduates  of  extremely  lower  and  upper  ends  of  mental  categories  than  from 
the  middle  mental  categories  (see  table  G-3). 


TABLE  G-3.  ABSENT  TAS  RATINGS  BY 

ABILITY  LEVEL 

Original 

Sample 

Size 

Absent 

TAS  Ratings 

AFQT 

Percentile 

Range 

Mental  Category 

N 

N 

% 

93-99 

1 

228 

113 

50 

65-92 

2 

1211 

503 

42 

49-64 

3U 

1608 

586 

36 

31-48 

3L 

828 

341 

41 

21-30 

4U 

36 

22 

61 

10-20 

4L 

1 

0 

0 

Missing  AFQT 

94 

26 

28 

Total 

4006 

1591 

40 

3.  Training  Task.  More  TAS  ratings  were  absent  from  graduates  who 
attended  courses  that  taught  a  relatively  smaller  percentage  of  fact  or 
rule  training  tasks  than  those  that  taught  a  larger  percentage  of  these  tasks 
(see  table  G-4). 

TABLE  G-4.  ABSENT  TAS  RATINGS  BY  TRAINING  TASK 


Percent  Task 

Taught 

in  Courses 

0 

1 

-25 

26 

-50 

51-75 

76-100 

Task 

N  X 

N 

% 

N 

X 

N  X 

N  % 

Fact 

1036  34 

555 

56 

- 

- 

- 

- 

Rule 

733  40 

670 

46 

133 

31 

23  12 

32  29 

i 
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CRITERION:  PRESENCE  VS.  ABSENCE  OF  TINE  TO  COMPLETE  (STUDENT  CONTACT  HOURS 
BASED  ON  SCHOOL  RECORDS) 

Only  main  effects  (and  no  Interactions)  were  found  in  the  regression 
analysis  presented  in  table  G-5. 


TABLE  G-5.  SIGNIFICANT  RESULTS  FROM  REGRESSION  ANALYSES  OF  ABSENT 
TIME  TO  COMPLETE  THE  COURSE  (CONTACT  HOURS)  DATA 


Step 

Variables 

B 

F 

OF 

R2 

Increment 

Overall  £_ 

OF 

1 

II  vs  Cl 

(Tl) 

-.02 

11.66 

1/4003 

.0239 

49.18 

2/4003 

SP  vs  CMI 

(T2) 

-.05 

72.16 

1/4003 

2 

Category 

(12) 

-.002 

8.41 

1/3998 

.0528 

228.64 

5/3998 

Procedure 

(13) 

O 

o 

• 

1 

111.99 

1/3998 

Rule 

(14) 

-.004 

189.64 

1/3998 

The  following  findings  are  listed  by  significant  predictor  variables. 

1.  Method  of  Instruction.  A  larger  percentage  of  time  to  complete 
data  was  absent  from  SP  and  CMI  graduates  than  GP  graduates.  Time  to 
complete  data  were  absent  more  from  SP  graduates  than  from  CMI  graduates 
(see  table  G-6). 


TABLE  G-6.  ABSENT  TIME  TO  COMPLETE  DATA  BY  METHOD  OF  INSTRUCTION 


Original 

Sample 

Size 

Absent 

Time  to  Complete 

N 

N 

* 

SP 

1451 

168 

12 

CMI 

823 

20 

2 

GP 

1732 

68 

4 

Total  4006 

256 

6 
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2.  Traininq  Task.  More  time  to  complete  data  were  absent  from  graduates 
who  attended  courses  that  taught  a  relatively  smaller  percentage  of  category, 
procedure,  or  rule  training  tasks  than  those  that  taught  a  larger  percentage  of 
those  tasks  (see  table  G-7). 


TABLE  G-7.  ABSENT  TIME  TO  COMPLETE  DATA  BY  TRAINING  TASK 


J"'--  ‘  - - -  1 

Percent  Task  Taught  in  Courses  { 

0  1-25  26-50  51-75  76-100  I 


Task 

N 

% 

N 

% 

N 

% 

N 

% 

N 

% 

Category 

26 

4 

219 

7 

11 

14 

- 

- 

- 

- 

Procedure 

- 

68 

9 

1 

1 

141 

15 

46 

2 

Rule 

128 

7 

105 

7 

23 

5 

- 

- 

- 

- 

CRITERION:  PRESENCE  VS.  ABSENCE  OF  END-OF-COURSE  GRADES 

Only  main  effects  (and  no  interactions)  were  found  in  the  regression 
analy' >s  presented  in  table  G-8. 


TABLE  G-8.  SIGNIFICANT  RESULTS  FROM  REGRESSION  ANALYSES  OF 
ABSENT  END-OF-COURSE  GRADES 


Step 

Variables 

B 

F 

DF 

R2 

Increment 

Overall  F 

- 

1 

II  vs  Cl 

(Tl) 

.14 

196.28 

1/4003 

.1197 

272.38 

2/4003  | 

SP  vs  CM I 

(T  2) 

-.21 

440.00 

1/4003 

J 

i 

2 

Category 

(12) 

-.01 

270.82 

1/3998 

.6453 

337.13 

5/3998 

Procedure 

(13) 

-.01 

1242.45 

1/3998 

( 

Rule 

(14) 

-.01 

692.24 

1/3998 
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The  following  findings  are  listed  by  significant  predictor  variables. 

1.  Method  of  Instruction.  A  larger  percentage  of  end-of-course 
grades  was  absent  from  GP  graduates  than  from  both  SP  and  CMI  graduates. 
End-of-course  grades  were  absent  more  from  SP  than  from  CMI  graduates  (see 
table  G-9). 


TABLE  G-9.  ABSENT  END-OF-COURSE  GRADES  BY  METHOD  OF  INSTRUCTION 


Original 

Sample 

Size 

Absent 

End-of-Course 

Grades 

N 

N 

% 

SP 

1451 

867 

60 

CMI 

823 

131 

16 

GP 

1732 

964 

56 

Total 

4006 

1962 

49 

2.  Training  Task.  More  end  of  course  grades  were  absent  from 
graduates  who  attended  courses  that  taught  a  relatively  larger  percentage  of 
procedure  or  rule  tasks,  or  a  smaller  percentage  of  category  tasks  than 
courses  that  taught  smaller  or  larger  amounts  of  these  tasks,  respectively 
(see  table  G-10). 


TABLE  G-10.  ABSENT  END-OF-COURSE  GRADES  BY  TRAINING  TASK 


Percent 

Task  Taught 

in  Courses 

0 

1- 

■25 

26- 

-50 

51 

-75 

76-100 

Task 

N 

% 

N 

% 

N 

% 

N 

51! 

N  % 

Category 

97 

16 

1854 

56 

11 

14 

- 

- 

- 

Procedure 

- 

- 

253 

32 

2 

1 

374 

39 

1333  65 

Rule 

1136 

62 

387 

27 

254 

59 

185 

100 

- 
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